-- Logs begin at Thu 2022-10-20 05:17:36 UTC, end at Thu 2022-10-20 06:51:41 UTC. -- Oct 20 05:50:48.443137 dv-cinder-861566-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 20 05:50:48.451760 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 20 05:50:48.453729 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Oct 20 05:50:48 2022] *** Oct 20 05:50:48.453729 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 20 05:50:48.453729 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 20 05:50:48.453729 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: nodename: dv-cinder-861566-3-iscsi Oct 20 05:50:48.453729 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: machine: x86_64 Oct 20 05:50:48.453729 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: clock source: unix Oct 20 05:50:48.453729 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: pcre jit disabled Oct 20 05:50:48.453729 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: detected number of CPU cores: 4 Oct 20 05:50:48.453729 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: current working directory: / Oct 20 05:50:48.453729 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: detected binary path: /usr/bin/uwsgi-core Oct 20 05:50:48.454263 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: your processes number limit is 47680 Oct 20 05:50:48.454263 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: your memory page size is 4096 bytes Oct 20 05:50:48.454263 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: detected max file descriptor number: 2048 Oct 20 05:50:48.454263 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: lock engine: pthread robust mutexes Oct 20 05:50:48.454263 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: thunder lock: enabled Oct 20 05:50:48.454263 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 20 05:50:48.454263 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 20 05:50:48.493557 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: Python main interpreter initialized at 0x556c8a4069c0 Oct 20 05:50:48.493557 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: python threads support enabled Oct 20 05:50:48.493557 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: your server socket listen backlog is limited to 100 connections Oct 20 05:50:48.493557 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: your mercy for graceful operations on workers is 90 seconds Oct 20 05:50:48.493912 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: mapped 403077 bytes (393 KB) for 2 cores Oct 20 05:50:48.493994 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: *** Operational MODE: preforking *** Oct 20 05:50:48.494247 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: *** uWSGI is running in multiple interpreter mode *** Oct 20 05:50:48.494247 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: spawned uWSGI master process (pid: 81341) Oct 20 05:50:48.494448 dv-cinder-861566-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 20 05:50:48.496388 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: spawned uWSGI worker 1 (pid: 81354, cores: 1) Oct 20 05:50:48.496388 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: spawned uWSGI worker 2 (pid: 81355, cores: 1) Oct 20 05:50:48.496388 dv-cinder-861566-3-iscsi devstack@keystone.service[81341]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 20 05:50:52.735944 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 20 05:50:52.735944 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: from cryptography.utils import int_from_bytes Oct 20 05:50:52.736774 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 20 05:50:52.736774 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: from cryptography.utils import int_from_bytes Oct 20 05:50:52.738004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 20 05:50:52.738004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: from cryptography.utils import int_from_bytes Oct 20 05:50:52.738569 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 20 05:50:52.738569 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: from cryptography.utils import int_from_bytes Oct 20 05:50:52.787082 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 20 05:50:52.787277 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 20 05:50:52.787444 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] command line args: [] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 20 05:50:52.787890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 20 05:50:52.788067 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ================================================================================ {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 20 05:50:52.788327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] admin_token = **** {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.788572 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] config_dir = [] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.788787 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.788968 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] config_source = [] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.789146 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] debug = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.789342 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.789983 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.790121 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 20 05:50:52.790319 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 20 05:50:52.790508 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] command line args: [] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 20 05:50:52.790713 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.790804 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 20 05:50:52.790945 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ================================================================================ {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 20 05:50:52.791200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] admin_token = **** {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.791467 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] config_dir = [] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.791581 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.791649 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.791751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.791856 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] config_source = [] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.791970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] debug = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.792181 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.792434 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.792534 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] list_limit = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.792610 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.792679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] log_config_append = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.792940 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.793037 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.793111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] log_dir = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.793178 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.793248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] log_file = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.793310 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] list_limit = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.793533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.793632 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] log_config_append = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.793632 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.793747 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.793820 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] log_dir = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.794017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.794121 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] log_file = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.794191 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.794256 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.794315 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.794534 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.794632 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.794697 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.794763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.794984 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.795094 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.795166 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.795234 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.795295 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.795495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.795603 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.795676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.795738 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.795941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.796039 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.796109 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.796176 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.796373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.796573 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.796763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.796869 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.796869 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.797349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.797557 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.797750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] publish_errors = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.797946 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.798065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.798065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.798065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.798472 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.798472 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] publish_errors = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.798623 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.798692 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.798925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.799059 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.799210 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] standard_threads = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.799351 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.799518 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.799669 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.799801 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.799936 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.800073 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.800073 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] use_journal = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.800479 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] standard_threads = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.800629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] use_json = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.800797 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.800992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] use_stderr = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.800992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] use_syslog = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.801318 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.801318 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.801586 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.801716 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] use_journal = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.801857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.801998 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] use_json = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.802168 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.802297 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] use_stderr = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.802425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.802554 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] use_syslog = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.802688 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.802828 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 05:50:52.802959 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.802959 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.803199 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.803327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.803491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.803661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.803812 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.803950 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.804082 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.804217 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.804353 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.804491 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.804620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.804770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.804911 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.805041 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.805175 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.805313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.805449 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.805589 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.805718 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.805846 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.805976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.806113 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.806249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.806378 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.806513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.806642 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.806770 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.806906 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.807036 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.807164 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.807299 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.807450 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.807588 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.807789 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.807924 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.808060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.808190 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.808318 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.808454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.808590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.808778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.808913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.809042 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.809172 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.809323 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.809454 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.809454 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.809702 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.809831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.809959 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.810096 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.810226 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.810355 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.810483 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.810616 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.810745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.810881 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.811027 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.811157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.811286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.811424 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.811553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.811693 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.811829 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.811957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.812084 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.812253 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.812419 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.812562 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.812707 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.812852 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.812992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.813122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.813250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.813387 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.813517 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.813651 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.813780 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.813908 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.814036 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.814238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.814370 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.814498 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.814626 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.814771 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.814907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.815044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.815180 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.815322 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.815452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.815580 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.815708 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.815847 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.816499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.816635 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.816823 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.816955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.817083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.817339 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.818219 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.818419 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.818617 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.818811 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.819005 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.819242 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.819409 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.819602 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.819865 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.820064 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.820297 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.820466 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.820716 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.820888 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.821122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.821290 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.821483 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.821718 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.821887 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.822113 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.822334 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.822500 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.822732 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.822899 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.823133 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.823299 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.823476 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.823654 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.823792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.823930 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.824101 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.824238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.824238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.824491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.824491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.824796 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.824943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.825074 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.825203 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.825351 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.825483 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.825612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.825750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.825880 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.826011 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.826170 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.826304 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.826434 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.826571 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.826708 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.826844 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.826975 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.827111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.827251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.827389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.827520 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.827650 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.827778 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.827907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.828036 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.828174 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.828305 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.828434 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.828563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.828726 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.828894 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.829050 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.829183 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.829312 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.829440 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.829569 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.829703 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.829843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.829973 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.830102 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.830230 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.830359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.830488 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.830488 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.830808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.830949 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.831079 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.831209 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.831348 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.831487 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.831724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.831879 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.832015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.832145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.832275 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.832450 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.832589 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.832738 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.832884 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.833131 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.833296 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.833431 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.833569 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.833569 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.833821 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.833951 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.834080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.834209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.834348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] auth.external = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.834479 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.834608 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.834750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.834881 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.835056 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.835197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.835335 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.835476 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] auth.password = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.835476 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] auth.token = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.835722 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.835865 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.835997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.836128 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.836266 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] auth.external = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.836398 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.836529 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.836658 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.836848 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.836999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.837143 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.837275 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.837413 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] auth.password = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.837544 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.837675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] auth.token = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.837805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] credential.caching = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.837944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.838081 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.838219 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.838351 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.838482 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.838626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.838768 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.838906 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.839038 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.839167 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.839297 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.839426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.839564 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.839696 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.839825 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] credential.caching = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.839955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.840085 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.840214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.840353 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.840494 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.840632 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.840784 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.840916 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.841047 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.841188 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.841320 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.841450 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.841580 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.841719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.841857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.842000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.842132 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] federation.caching = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.842262 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.842443 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.842443 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.842696 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.842835 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.842966 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.843096 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.843226 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.843356 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.843487 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.843626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.843758 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.843888 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.844018 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.844169 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.844310 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] federation.caching = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.845096 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.845246 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.845378 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.845509 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.845639 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.845778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.845910 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.caching = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.846041 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.846171 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.846301 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.846432 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.846570 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.846712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.846843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.846975 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.847115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.847784 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.847995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.848130 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.848260 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.848391 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.848521 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.848651 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.848816 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.caching = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.848949 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.849091 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.849222 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.849353 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.849483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.849621 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.849753 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.849883 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.850014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.850144 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.850274 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.850412 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.850544 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.850674 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.850804 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.850959 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.851099 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.851240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.851372 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.851502 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.851631 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.851768 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.851899 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.852038 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.852180 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.852317 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.852497 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.852635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.852834 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.852978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.853110 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.853110 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.853356 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.853486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.853615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.853615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.853876 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.854005 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.854135 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.854265 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.854395 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.855244 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.855332 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.855399 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.855463 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.855525 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.855588 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.855659 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.855723 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.855784 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.855845 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.855905 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.855966 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.855966 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.856084 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.856145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.856205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.856265 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.856333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.856395 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.856456 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.856516 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.856576 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.856636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.856729 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.856796 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.856857 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.856917 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.856977 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.857036 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.857110 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.857175 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.857235 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.857303 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.857375 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.857436 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.857503 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.857565 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.857625 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.857693 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.857754 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.857814 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.857899 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.857974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.858053 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.858129 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.858201 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.858273 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.858353 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.858639 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.858725 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.858806 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.858806 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.858937 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.859017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.859092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.859164 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.859235 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.859306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.859306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.859441 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.859515 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.859595 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.859666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.859738 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.859825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.859907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.859981 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.860053 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.860124 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.860196 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.860267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.860347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.860421 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.860493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.860565 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.860637 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.860727 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.860811 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.860887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.860959 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.861031 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.861103 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.861174 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.861255 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.861335 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.861414 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.861486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.861557 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.861628 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.861708 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.861782 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.861854 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.861925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.861996 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.862068 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.862147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.862222 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.862293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.862365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.862436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.862507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.862586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.862661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.862733 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.862805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.862974 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.863043 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.863124 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.863198 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.863270 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.863342 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.863413 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.863484 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.863564 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.863639 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] resource.caching = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.863711 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.863782 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.863853 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.863930 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.864012 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.864087 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.864159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.864231 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.864302 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.864373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.864461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.864536 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.864613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.864706 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.864783 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.864855 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.864943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.865018 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.865089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.865160 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] role.caching = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.865232 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.865304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] role.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.865383 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] resource.caching = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.865457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.865529 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.865601 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.865673 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.865819 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.865896 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.865969 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.866041 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.866112 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.866184 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.866263 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.866338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.866409 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.866481 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.866552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.866623 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.866702 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.866776 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.866848 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.866919 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] role.caching = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.866990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.867069 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] role.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.867150 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.867225 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.867298 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.867369 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.867440 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.867512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.867512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.867651 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.867651 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.867787 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.867860 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.867932 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.867932 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.868071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.868143 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.868215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.868215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.868344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.868344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.868482 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.868482 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.868610 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.868680 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.868680 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.869200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.869277 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.869349 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.869421 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.869493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.869581 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.869581 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.869717 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.869794 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.869867 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.869939 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.870018 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.870092 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.870164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.870236 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.870307 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.870379 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.870457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.870532 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] token.caching = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.870604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.870675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.870745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.870816 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.870895 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.870969 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.871041 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.871112 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.871189 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.871262 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.871341 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.871415 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.871486 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.871558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.871629 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.871702 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.871781 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.871855 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.871927 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.871998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.872069 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.872141 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.872220 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.872293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.872364 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.872442 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.872515 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] token.caching = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.872587 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.872665 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.872770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.872846 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.872928 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.873003 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.873074 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.873146 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.873224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.connection = **** {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.873299 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.873370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.873441 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.873512 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.873584 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.873663 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.873737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.873808 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.873879 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.873950 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.874021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.874100 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.874175 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.874246 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.874318 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.874388 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.874459 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.874538 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.874538 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.874669 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.874669 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.874795 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.874795 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.874935 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.875007 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.875087 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.875161 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.875233 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.connection = **** {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.875304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.875375 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.875446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.875525 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.875600 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.875672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.875744 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.875815 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.875887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 20 05:50:52.875966 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.875966 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.875966 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.875966 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.875966 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.875966 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.875966 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.876410 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.876410 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.876536 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.876710 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.876947 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.877188 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.877429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 05:50:52.877599 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 20 05:50:52.904657 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:52.904919 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:52.905139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:52.907444 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:52.907679 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:52.907907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:52.913545 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:52.916520 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:52.942146 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:52.942366 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:52.943714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:52.943947 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:52.994668 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:52.995254 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.042709 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.042938 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.044486 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.044743 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.197772 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.198002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.198224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.198437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.198611 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.198727 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.198848 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.198935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.199075 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.199295 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.199514 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.199730 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.221834 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.222062 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.222157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.222684 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.222789 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.222789 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.222917 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.222990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.223063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.223144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.223259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.223342 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.223482 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.223700 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.234238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.234504 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.247364 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.247556 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.247782 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.247893 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.248006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.248093 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.248228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.248312 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.248443 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.248538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.248659 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.248763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.248904 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.249121 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 05:50:53.265903 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.266110 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 20 05:50:53.266418 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.266503 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.266618 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 20 05:50:53.266700 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 20 05:50:53.266879 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.266963 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.267135 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 20 05:50:53.267216 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.267387 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.267477 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.267640 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.267723 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.267902 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.268161 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.277186 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.277464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.277729 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.278014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.278095 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.278300 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.278571 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.278853 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.285397 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.286585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.288256 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.288539 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.288854 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.289134 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.289411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.289492 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.289667 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.289761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.289931 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.290037 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.290201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.290320 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.290467 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.290601 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.290733 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.290862 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.290995 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.291133 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.291408 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.291684 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.291957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.292226 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.314116 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.315552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.315640 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.315842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.317406 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.317696 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.320588 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.320900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.321178 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.322345 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.322633 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.322917 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.327401 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.327685 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.329215 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.329507 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.333342 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.333646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.333927 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.334209 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.334485 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.334760 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.335033 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.335300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.335601 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.335901 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.336191 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.336482 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.336788 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.337065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.337337 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.337605 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.351096 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.351374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.353700 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.353984 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.355192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.357832 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.358398 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.358700 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.361067 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.361367 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.362936 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.366421 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.366626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.369568 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.369847 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.370146 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.370427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.370707 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.370987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.371268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.372753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.373058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.373344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.373630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.373907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.374186 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.387155 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.387442 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.387710 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.387972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.390338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.390630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.390900 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.391162 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.394751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.396266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.397807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.398082 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.398324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.399836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.400822 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.401112 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.401362 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.401455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.401641 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.401724 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.402022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.404211 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.404498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.404833 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.405190 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.405574 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.413516 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.413796 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.414073 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.414357 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.414732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.415023 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.415300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.417445 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.417742 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.418026 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.418316 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.418700 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.418990 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.419269 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.431009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.433886 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.435520 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.436741 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.438310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.439387 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.439895 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.440175 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.440573 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.442633 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.444070 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.445602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.445889 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.446291 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.446961 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.448398 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.451271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.451664 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.451946 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.452221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.452809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.454109 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.457178 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.457656 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.457943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.458219 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.460051 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.460337 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.460622 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.466014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.466310 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.466594 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.467392 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.467673 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.467952 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.468227 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.468497 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.468805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.469080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.469349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.469613 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.469884 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.470158 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.470420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.470679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.470956 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.474225 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 05:50:53.474520 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.474799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.475081 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.475351 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.475619 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.475890 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.476171 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.476463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.476758 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.477032 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.477299 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.477562 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.477821 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 05:50:53.505507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556c8a4069c0 pid: 81354 (default app) Oct 20 05:50:53.508974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f528e-5d8a-4861-a9b3-3b6fd2fae28f None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:50:53.509258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f528e-5d8a-4861-a9b3-3b6fd2fae28f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:50:53.509534 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f528e-5d8a-4861-a9b3-3b6fd2fae28f None None] PATH_INFO: `/v3` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:50:53.512797 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556c8a4069c0 pid: 81355 (default app) Oct 20 05:50:53.513636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Thu Oct 20 05:50:53 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:00.266160 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548eb439-8c3f-477b-97f4-c6f5e4e03875 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:00.266673 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548eb439-8c3f-477b-97f4-c6f5e4e03875 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:00.266781 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548eb439-8c3f-477b-97f4-c6f5e4e03875 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:00.272043 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:51:00.277465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5d02a-99bc-401a-82ec-7bcd7c25f931 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:00.277875 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5d02a-99bc-401a-82ec-7bcd7c25f931 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:00.278007 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5d02a-99bc-401a-82ec-7bcd7c25f931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:00.347692 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: INFO dbcounter [None req-e7d5d02a-99bc-401a-82ec-7bcd7c25f931 None None] Registered counter for database keystone Oct 20 05:51:00.366295 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG oslo_db.sqlalchemy.engines [None req-e7d5d02a-99bc-401a-82ec-7bcd7c25f931 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81354) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 20 05:51:00.372342 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writer thread running {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 20 05:51:00.549094 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG passlib.handlers.bcrypt [None req-e7d5d02a-99bc-401a-82ec-7bcd7c25f931 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81354) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 20 05:51:00.549403 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG passlib.handlers.bcrypt [None req-e7d5d02a-99bc-401a-82ec-7bcd7c25f931 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81354) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 20 05:51:00.606961 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-e7d5d02a-99bc-401a-82ec-7bcd7c25f931 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:00.608541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e7d5d02a-99bc-401a-82ec-7bcd7c25f931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:00.734203 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 20 05:51:01.176118 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548eb439-8c3f-477b-97f4-c6f5e4e03875 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:01.246331 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: INFO dbcounter [None req-548eb439-8c3f-477b-97f4-c6f5e4e03875 None None] Registered counter for database keystone Oct 20 05:51:01.268044 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG oslo_db.sqlalchemy.engines [None req-548eb439-8c3f-477b-97f4-c6f5e4e03875 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81355) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 20 05:51:01.274485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writer thread running {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 20 05:51:01.393917 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548eb439-8c3f-477b-97f4-c6f5e4e03875 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:01.398320 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960477d-acfb-4fbe-b1ea-ad4674fbcfac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:01.398943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960477d-acfb-4fbe-b1ea-ad4674fbcfac admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:01.399224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960477d-acfb-4fbe-b1ea-ad4674fbcfac admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:01.399498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960477d-acfb-4fbe-b1ea-ad4674fbcfac admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:01.407640 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960477d-acfb-4fbe-b1ea-ad4674fbcfac admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:01.459126 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8960477d-acfb-4fbe-b1ea-ad4674fbcfac admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 20 05:51:01.459408 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960477d-acfb-4fbe-b1ea-ad4674fbcfac admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:01.468769 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Thu Oct 20 05:51:01 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 05:51:01.474978 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5d02a-99bc-401a-82ec-7bcd7c25f931 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:01.492379 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5d02a-99bc-401a-82ec-7bcd7c25f931 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:01.496826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b5eb1c-d2f8-456d-8308-03c7942b8bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:01.497921 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5eb1c-d2f8-456d-8308-03c7942b8bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:01.498200 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5eb1c-d2f8-456d-8308-03c7942b8bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:01.498471 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5eb1c-d2f8-456d-8308-03c7942b8bf2 admin admin] PATH_INFO: `/v3/projects/edda8534137f405d818dbc74b7391c73` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:01.500091 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b5eb1c-d2f8-456d-8308-03c7942b8bf2 admin admin] RBAC: Authorizing `identity:get_project(project_id=edda8534137f405d818dbc74b7391c73)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:01.555953 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: WARNING keystone.common.rbac_enforcer.enforcer [None req-29b5eb1c-d2f8-456d-8308-03c7942b8bf2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 20 05:51:01.556300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b5eb1c-d2f8-456d-8308-03c7942b8bf2 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:01.560820 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Thu Oct 20 05:51:01 2022] GET /identity/v3/projects/edda8534137f405d818dbc74b7391c73 => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:03.650799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6bba1-7849-4707-b8d2-a220482ad2a0 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:03.651292 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6bba1-7849-4707-b8d2-a220482ad2a0 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:03.651378 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6bba1-7849-4707-b8d2-a220482ad2a0 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:03.655457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:03.660765 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f301fb-fd65-4ad0-a3ce-e971b2d76f76 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:03.661171 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f301fb-fd65-4ad0-a3ce-e971b2d76f76 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:03.661307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f301fb-fd65-4ad0-a3ce-e971b2d76f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:03.681253 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-15f301fb-fd65-4ad0-a3ce-e971b2d76f76 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:03.682734 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-15f301fb-fd65-4ad0-a3ce-e971b2d76f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:03.690390 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 20 05:51:03.970614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6bba1-7849-4707-b8d2-a220482ad2a0 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:03.982766 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6bba1-7849-4707-b8d2-a220482ad2a0 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:03.986966 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d973810-c421-4332-b14c-fd1c581e993d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:03.987562 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d973810-c421-4332-b14c-fd1c581e993d admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:03.987843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d973810-c421-4332-b14c-fd1c581e993d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:03.988227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d973810-c421-4332-b14c-fd1c581e993d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:03.995872 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d973810-c421-4332-b14c-fd1c581e993d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:03.997671 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d973810-c421-4332-b14c-fd1c581e993d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:04.006987 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Thu Oct 20 05:51:03 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 05:51:04.012588 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f301fb-fd65-4ad0-a3ce-e971b2d76f76 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:04.022149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f301fb-fd65-4ad0-a3ce-e971b2d76f76 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:04.026432 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d337b-c06d-4862-a8f7-c387906cf84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:04.027054 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d337b-c06d-4862-a8f7-c387906cf84c admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:04.027334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d337b-c06d-4862-a8f7-c387906cf84c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:04.027606 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d337b-c06d-4862-a8f7-c387906cf84c admin admin] PATH_INFO: `/v3/users/6f317beb484644609d07d666527f760d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:04.029032 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d337b-c06d-4862-a8f7-c387906cf84c admin admin] RBAC: Authorizing `identity:get_user(user_id=6f317beb484644609d07d666527f760d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:04.030844 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d337b-c06d-4862-a8f7-c387906cf84c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:04.032345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Thu Oct 20 05:51:04 2022] GET /identity/v3/users/6f317beb484644609d07d666527f760d => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:06.224042 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6ba32-8596-4dc1-9217-56539f07413b None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:06.224632 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6ba32-8596-4dc1-9217-56539f07413b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:06.224632 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6ba32-8596-4dc1-9217-56539f07413b None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:06.225928 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:06.231082 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7685e0-1932-403f-be24-8850867cec01 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:06.231513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7685e0-1932-403f-be24-8850867cec01 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:06.231633 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7685e0-1932-403f-be24-8850867cec01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:06.250056 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-df7685e0-1932-403f-be24-8850867cec01 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:06.251228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-df7685e0-1932-403f-be24-8850867cec01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:06.262559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1861e-cf99-4f3b-b8a1-f06fb415da09 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:06.262841 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1861e-cf99-4f3b-b8a1-f06fb415da09 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:06.263110 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1861e-cf99-4f3b-b8a1-f06fb415da09 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:06.264412 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:51:06.270408 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fbb79-2140-42b5-8811-4b20d2df668d None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:06.270712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fbb79-2140-42b5-8811-4b20d2df668d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:06.270987 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fbb79-2140-42b5-8811-4b20d2df668d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:06.312749 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG passlib.handlers.bcrypt [None req-6c1fbb79-2140-42b5-8811-4b20d2df668d None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81355) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 20 05:51:06.313142 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG passlib.handlers.bcrypt [None req-6c1fbb79-2140-42b5-8811-4b20d2df668d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81355) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 20 05:51:06.335496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:06.362317 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-6c1fbb79-2140-42b5-8811-4b20d2df668d None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:06.364011 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6c1fbb79-2140-42b5-8811-4b20d2df668d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:06.462443 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 05:51:06.601798 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7685e0-1932-403f-be24-8850867cec01 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:06.626725 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:06.704298 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7685e0-1932-403f-be24-8850867cec01 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:06.728248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fbb79-2140-42b5-8811-4b20d2df668d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:06.737636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac5702-b22a-42d8-9f96-4d1ce7997e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:06.738409 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac5702-b22a-42d8-9f96-4d1ce7997e6d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:06.738843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac5702-b22a-42d8-9f96-4d1ce7997e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:06.739116 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac5702-b22a-42d8-9f96-4d1ce7997e6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:06.749935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac5702-b22a-42d8-9f96-4d1ce7997e6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:06.759221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-58ac5702-b22a-42d8-9f96-4d1ce7997e6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:51:06.759221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:06.759221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:06.759221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:06.759221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:06.759221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:06.759221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:06.759221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:06.759221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:06.759221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:06.759221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:06.759221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:06.759221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:06.759221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:06.759221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:06.759221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:06.760330 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:06.760330 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:06.760330 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:06.760330 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:06.760330 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:06.760330 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:06.760330 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:06.760330 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:06.760330 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:06.760330 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:06.760330 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:06.760330 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:06.760330 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:06.760330 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:06.760330 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:06.760330 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:06.761214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:06.761214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:06.761214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:06.761214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:06.761214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:06.761214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:06.761214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:06.761214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:06.761214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:06.761214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:06.761214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:51:06.761214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:51:06.761214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Oct 20 05:51:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:51:06.765919 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac5702-b22a-42d8-9f96-4d1ce7997e6d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:06.829944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fbb79-2140-42b5-8811-4b20d2df668d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:06.832871 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:06.847231 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:06.865487 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac5702-b22a-42d8-9f96-4d1ce7997e6d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:06.867615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e28d9-cbfc-454b-befb-9f61ecd6099b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:06.868470 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e28d9-cbfc-454b-befb-9f61ecd6099b None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:06.868784 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e28d9-cbfc-454b-befb-9f61ecd6099b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:06.869058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e28d9-cbfc-454b-befb-9f61ecd6099b None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:06.869999 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e28d9-cbfc-454b-befb-9f61ecd6099b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:06.871420 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e28d9-cbfc-454b-befb-9f61ecd6099b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:06.883657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.sql.core [None req-4d3e28d9-cbfc-454b-befb-9f61ecd6099b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 20 05:51:06.883657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 20 05:51:06.883657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [parameters: {'id': '56a19cd39e1242788c0ce26678635b38', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 20 05:51:06.883657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81355) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 20 05:51:06.900160 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feacdc3b-1d09-422a-95fe-180d11928870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:06.900832 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feacdc3b-1d09-422a-95fe-180d11928870 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:06.901109 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feacdc3b-1d09-422a-95fe-180d11928870 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:06.901377 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feacdc3b-1d09-422a-95fe-180d11928870 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:06.902041 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feacdc3b-1d09-422a-95fe-180d11928870 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:06.902343 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feacdc3b-1d09-422a-95fe-180d11928870 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:06.902499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-4d3e28d9-cbfc-454b-befb-9f61ecd6099b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 20 05:51:06.902499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:06.902499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 05:51:06.902499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 05:51:06.902499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 05:51:06.902499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 05:51:06.902499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 05:51:06.902499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 05:51:06.902499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 05:51:06.902499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application conn.query(q) Oct 20 05:51:06.902499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 05:51:06.902499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 05:51:06.902499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 05:51:06.902499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application result.read() Oct 20 05:51:06.903359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 05:51:06.903359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 05:51:06.903359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 05:51:06.903359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 05:51:06.903359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 05:51:06.903359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 05:51:06.903359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 05:51:06.903359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 05:51:06.903359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 20 05:51:06.903359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:51:06.903359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 20 05:51:06.903359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:51:06.903359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:06.903359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 20 05:51:06.903359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 20 05:51:06.903359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 20 05:51:06.903359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 20 05:51:06.903359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 05:51:06.904460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application next(self.gen) Oct 20 05:51:06.904460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 20 05:51:06.904460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application yield resource Oct 20 05:51:06.904460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 05:51:06.904460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application next(self.gen) Oct 20 05:51:06.904460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 20 05:51:06.904460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self.session.rollback() Oct 20 05:51:06.904460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 20 05:51:06.904460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self.force_reraise() Oct 20 05:51:06.904460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 20 05:51:06.904460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise self.value Oct 20 05:51:06.904460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 20 05:51:06.904460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 20 05:51:06.904460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 20 05:51:06.904460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application session.commit() Oct 20 05:51:06.904460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 20 05:51:06.904460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 20 05:51:06.905620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 20 05:51:06.905620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._prepare_impl() Oct 20 05:51:06.905620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 20 05:51:06.905620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self.session.flush() Oct 20 05:51:06.905620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 20 05:51:06.905620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._flush(objects) Oct 20 05:51:06.905620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 20 05:51:06.905620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 20 05:51:06.905620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 20 05:51:06.905620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application compat.raise_( Oct 20 05:51:06.905620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 05:51:06.905620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception Oct 20 05:51:06.905620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 20 05:51:06.905620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application flush_context.execute() Oct 20 05:51:06.905620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 20 05:51:06.905620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rec.execute(self) Oct 20 05:51:06.905620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 20 05:51:06.906493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 20 05:51:06.906493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 20 05:51:06.906493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 20 05:51:06.906493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 20 05:51:06.906493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 20 05:51:06.906493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 20 05:51:06.906493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 20 05:51:06.906493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 20 05:51:06.906493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 20 05:51:06.906493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 20 05:51:06.906493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 20 05:51:06.906493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 20 05:51:06.906493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 20 05:51:06.906493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 20 05:51:06.906493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 20 05:51:06.907248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 05:51:06.907248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception Oct 20 05:51:06.907248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 05:51:06.907248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 05:51:06.907248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 05:51:06.907248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 05:51:06.907248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 05:51:06.907248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 05:51:06.907248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 05:51:06.907248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application conn.query(q) Oct 20 05:51:06.907248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 05:51:06.907248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 05:51:06.907248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 05:51:06.907248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application result.read() Oct 20 05:51:06.907248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 05:51:06.907248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 05:51:06.907248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 05:51:06.908054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 05:51:06.908054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 05:51:06.908054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 05:51:06.908054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 05:51:06.908054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 05:51:06.908054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 20 05:51:06.908054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 20 05:51:06.908054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [parameters: {'id': '56a19cd39e1242788c0ce26678635b38', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 20 05:51:06.908054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 20 05:51:06.908054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:51:06.908054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:51:06.908054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:51:06.908054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:06.908054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 20 05:51:06.908054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 20 05:51:06.909518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 20 05:51:06.909518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 20 05:51:06.909518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 20 05:51:06.909518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:51:06.909518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:51:06.909518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:51:06.909518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:06.909518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:06.909518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:06.909518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:06.909518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:06.909518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:06.909518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:06.909518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:06.909518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:06.909518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:06.909518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:06.911513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 20 05:51:06.911513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 20 05:51:06.911513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:06.911513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:06.911513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 20 05:51:06.911513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 20 05:51:06.911513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:06.911513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:06.911513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 20 05:51:06.911513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 20 05:51:06.911513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 20 05:51:06.911513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:51:06.911513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 8/15] 10.222.0.20 () {60 vars in 1137 bytes} [Thu Oct 20 05:51:06 2022] POST /identity/v3/domains => generated 190 bytes in 176 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:51:06.912477 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feacdc3b-1d09-422a-95fe-180d11928870 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:06.912566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e28d9-cbfc-454b-befb-9f61ecd6099b None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:06.919110 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 8/16] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:51:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:06.925107 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feacdc3b-1d09-422a-95fe-180d11928870 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:07.007743 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e28d9-cbfc-454b-befb-9f61ecd6099b None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:07.023844 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feacdc3b-1d09-422a-95fe-180d11928870 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:07.040575 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e2a9c-ee25-497b-b813-e97d3ede537f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:07.041459 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e2a9c-ee25-497b-b813-e97d3ede537f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:07.041734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e2a9c-ee25-497b-b813-e97d3ede537f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:07.041996 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e2a9c-ee25-497b-b813-e97d3ede537f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:07.055254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66b253-7b8d-4891-90f2-eafb7c39e5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:07.055875 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66b253-7b8d-4891-90f2-eafb7c39e5e8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:07.056147 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66b253-7b8d-4891-90f2-eafb7c39e5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:07.056412 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66b253-7b8d-4891-90f2-eafb7c39e5e8 None admin] PATH_INFO: `/v3/users/6f317beb484644609d07d666527f760d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:07.057566 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66b253-7b8d-4891-90f2-eafb7c39e5e8 None admin] RBAC: Authorizing `identity:get_user(user_id=6f317beb484644609d07d666527f760d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:07.058994 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66b253-7b8d-4891-90f2-eafb7c39e5e8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:07.059580 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e2a9c-ee25-497b-b813-e97d3ede537f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:07.060522 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 9/17] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:06 2022] GET /identity/v3/users/6f317beb484644609d07d666527f760d => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:07.061100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e2a9c-ee25-497b-b813-e97d3ede537f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:07.065891 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66b253-7b8d-4891-90f2-eafb7c39e5e8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:07.078072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-a14e2a9c-ee25-497b-b813-e97d3ede537f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:51:07.078072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:07.078072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:07.078072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:07.078072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:07.078072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:07.078072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:07.078072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:07.078072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:07.078072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:07.078072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:07.078072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:07.078072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:51:07.078072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:51:07.078072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:51:07.078072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:51:07.078960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:07.078960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:07.078960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:07.078960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:07.078960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:07.078960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:07.078960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:07.078960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:07.078960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:07.078960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:07.078960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:07.078960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:07.078960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:07.078960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:07.078960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:07.078960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:07.078960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:51:07.079758 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:51:07.079758 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:51:07.079758 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:51:07.079758 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 9/18] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:51:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:51:07.082850 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:07.085478 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e2a9c-ee25-497b-b813-e97d3ede537f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:07.107194 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:07.167255 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66b253-7b8d-4891-90f2-eafb7c39e5e8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:07.186918 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e2a9c-ee25-497b-b813-e97d3ede537f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:07.200014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef088f55-32a7-4373-a441-ee2fd71de4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:07.200648 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef088f55-32a7-4373-a441-ee2fd71de4ac None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:07.200951 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef088f55-32a7-4373-a441-ee2fd71de4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:07.201219 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef088f55-32a7-4373-a441-ee2fd71de4ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:07.202327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef088f55-32a7-4373-a441-ee2fd71de4ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:07.203762 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef088f55-32a7-4373-a441-ee2fd71de4ac None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:07.205515 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 10/19] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:51:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:07.211452 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef088f55-32a7-4373-a441-ee2fd71de4ac None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:07.224445 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe50f2-cd8a-4ebc-b5a2-4cc84bee9d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:07.225087 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe50f2-cd8a-4ebc-b5a2-4cc84bee9d3d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:07.225366 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe50f2-cd8a-4ebc-b5a2-4cc84bee9d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:07.225642 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe50f2-cd8a-4ebc-b5a2-4cc84bee9d3d None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:07.226273 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe50f2-cd8a-4ebc-b5a2-4cc84bee9d3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:07.226574 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe50f2-cd8a-4ebc-b5a2-4cc84bee9d3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:07.227944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe50f2-cd8a-4ebc-b5a2-4cc84bee9d3d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:07.245054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 10/20] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:07.295496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:07.320098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef088f55-32a7-4373-a441-ee2fd71de4ac None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:07.355980 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b00622-4a99-4bff-8c8e-60c3ed57dd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:07.356711 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b00622-4a99-4bff-8c8e-60c3ed57dd5a None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:07.356996 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b00622-4a99-4bff-8c8e-60c3ed57dd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:07.357271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b00622-4a99-4bff-8c8e-60c3ed57dd5a None admin] PATH_INFO: `/v3/domains/default/users/6f317beb484644609d07d666527f760d/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:07.359251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b00622-4a99-4bff-8c8e-60c3ed57dd5a None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=6f317beb484644609d07d666527f760d, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:07.360783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b00622-4a99-4bff-8c8e-60c3ed57dd5a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:07.479627 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 11/21] 10.222.0.20 () {58 vars in 1361 bytes} [Thu Oct 20 05:51:07 2022] PUT /identity/v3/domains/default/users/6f317beb484644609d07d666527f760d/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 05:51:09.593734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c930d-de0b-415a-b85c-85c5a6955f44 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:09.594215 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c930d-de0b-415a-b85c-85c5a6955f44 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:09.594215 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c930d-de0b-415a-b85c-85c5a6955f44 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:09.595945 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 11/22] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:09.602264 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157eb0e9-a72d-450b-9a39-5e740dc090b7 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:09.603078 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157eb0e9-a72d-450b-9a39-5e740dc090b7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:09.603552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157eb0e9-a72d-450b-9a39-5e740dc090b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:09.630100 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-157eb0e9-a72d-450b-9a39-5e740dc090b7 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:09.631914 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-157eb0e9-a72d-450b-9a39-5e740dc090b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:09.723625 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 12/23] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:10.146032 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e36a5-81fa-4892-a982-99fd43fee7fc None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:10.146405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e36a5-81fa-4892-a982-99fd43fee7fc None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:10.146579 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e36a5-81fa-4892-a982-99fd43fee7fc None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:10.147897 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 12/24] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:10.164105 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d600ac-4fe0-4c67-b825-518261104c69 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:10.164734 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d600ac-4fe0-4c67-b825-518261104c69 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:10.165231 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d600ac-4fe0-4c67-b825-518261104c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:10.190089 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a9d600ac-4fe0-4c67-b825-518261104c69 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:10.191767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a9d600ac-4fe0-4c67-b825-518261104c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:10.242591 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e36a5-81fa-4892-a982-99fd43fee7fc None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:10.287709 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:10.295275 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 13/25] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:10.366208 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54accc77-2fbb-4309-b79c-ce99a920284c None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:10.366833 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54accc77-2fbb-4309-b79c-ce99a920284c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:10.367335 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54accc77-2fbb-4309-b79c-ce99a920284c None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:10.372006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 14/26] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:51:10.382601 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1669e-4827-4380-85c0-199b5fba9937 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:10.383176 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1669e-4827-4380-85c0-199b5fba9937 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:10.383898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1669e-4827-4380-85c0-199b5fba9937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:10.417026 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:10.418556 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-99e1669e-4827-4380-85c0-199b5fba9937 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:10.420008 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-99e1669e-4827-4380-85c0-199b5fba9937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:10.450903 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e36a5-81fa-4892-a982-99fd43fee7fc None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:10.499148 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f072c-f573-4c28-97a0-e003f99b1a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:10.500157 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f072c-f573-4c28-97a0-e003f99b1a78 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:10.500440 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f072c-f573-4c28-97a0-e003f99b1a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:10.501014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f072c-f573-4c28-97a0-e003f99b1a78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:10.523887 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f072c-f573-4c28-97a0-e003f99b1a78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:10.525392 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f072c-f573-4c28-97a0-e003f99b1a78 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:10.545136 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 15/27] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 05:51:10.546405 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ff213-5898-43d8-8c08-b74c7e27116f None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:10.546691 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ff213-5898-43d8-8c08-b74c7e27116f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:10.546963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ff213-5898-43d8-8c08-b74c7e27116f None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:10.548207 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 16/28] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:10.557997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-f51f072c-f573-4c28-97a0-e003f99b1a78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:51:10.557997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:10.557997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:10.557997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:10.557997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:10.557997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:10.557997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:10.557997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:10.557997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:10.557997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:10.557997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:10.557997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:10.557997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:51:10.557997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:51:10.557997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:51:10.557997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:51:10.558877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:10.558877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:10.558877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:10.558877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:10.558877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:10.558877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:10.558877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:10.558877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:10.558877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:10.558877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:10.558877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:10.558877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:10.558877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:10.558877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:10.558877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:10.558877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:10.558877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:51:10.560456 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:51:10.560456 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:51:10.560456 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:51:10.560456 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 13/29] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:51:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:51:10.565100 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fb3b8-6afb-4b73-80b6-71503a4d3be2 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:10.565388 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fb3b8-6afb-4b73-80b6-71503a4d3be2 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:10.565668 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fb3b8-6afb-4b73-80b6-71503a4d3be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:10.569162 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f072c-f573-4c28-97a0-e003f99b1a78 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:10.594468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a59fb3b8-6afb-4b73-80b6-71503a4d3be2 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:10.595659 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a59fb3b8-6afb-4b73-80b6-71503a4d3be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:10.707007 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 17/30] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:10.708740 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fb3b8-6afb-4b73-80b6-71503a4d3be2 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:10.744823 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f072c-f573-4c28-97a0-e003f99b1a78 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:10.809067 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:10.819667 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e27d2a-2dac-4783-8f0a-6a3518ee7870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:10.820294 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e27d2a-2dac-4783-8f0a-6a3518ee7870 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:10.820581 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e27d2a-2dac-4783-8f0a-6a3518ee7870 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:10.820878 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e27d2a-2dac-4783-8f0a-6a3518ee7870 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:10.821528 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e27d2a-2dac-4783-8f0a-6a3518ee7870 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:10.821838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e27d2a-2dac-4783-8f0a-6a3518ee7870 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:10.824725 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e27d2a-2dac-4783-8f0a-6a3518ee7870 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:10.838355 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 14/31] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:10.847010 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e27d2a-2dac-4783-8f0a-6a3518ee7870 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:10.849042 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:10.955268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fb3b8-6afb-4b73-80b6-71503a4d3be2 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:11.028681 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097d684-ad14-4963-b738-77fd70bf589f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:11.029251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097d684-ad14-4963-b738-77fd70bf589f None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:11.029537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097d684-ad14-4963-b738-77fd70bf589f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:11.029807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097d684-ad14-4963-b738-77fd70bf589f None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:11.031626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a097d684-ad14-4963-b738-77fd70bf589f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:11.033181 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a097d684-ad14-4963-b738-77fd70bf589f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:11.064791 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 18/32] 10.222.0.20 () {60 vars in 1131 bytes} [Thu Oct 20 05:51:10 2022] POST /identity/v3/roles => generated 219 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:51:11.065611 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097d684-ad14-4963-b738-77fd70bf589f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:11.074040 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e27d2a-2dac-4783-8f0a-6a3518ee7870 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:11.184251 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a9d37-5d96-41b7-a850-de6c070f521c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:11.184917 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a9d37-5d96-41b7-a850-de6c070f521c None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:11.185197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a9d37-5d96-41b7-a850-de6c070f521c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:11.185473 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a9d37-5d96-41b7-a850-de6c070f521c None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:11.186131 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a9d37-5d96-41b7-a850-de6c070f521c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:11.187604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a9d37-5d96-41b7-a850-de6c070f521c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:11.268916 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097d684-ad14-4963-b738-77fd70bf589f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:11.310159 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:11.329419 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c3c1f8-9441-4f9d-9c02-89e83eaaa32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:11.330608 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3c1f8-9441-4f9d-9c02-89e83eaaa32e None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:11.331156 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3c1f8-9441-4f9d-9c02-89e83eaaa32e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:11.331644 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3c1f8-9441-4f9d-9c02-89e83eaaa32e None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:11.332574 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c3c1f8-9441-4f9d-9c02-89e83eaaa32e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:11.334361 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c3c1f8-9441-4f9d-9c02-89e83eaaa32e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:11.353596 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 19/33] 10.222.0.20 () {60 vars in 1131 bytes} [Thu Oct 20 05:51:11 2022] POST /identity/v3/roles => generated 225 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:51:11.354925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c3c1f8-9441-4f9d-9c02-89e83eaaa32e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:11.404483 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 15/34] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Oct 20 05:51:10 2022] POST /identity/v3/projects => generated 301 bytes in 558 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:51:11.492793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c3c1f8-9441-4f9d-9c02-89e83eaaa32e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:11.552016 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422897f-bf0e-428f-a5b7-cf3c061042b5 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:11.552381 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422897f-bf0e-428f-a5b7-cf3c061042b5 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:11.552559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422897f-bf0e-428f-a5b7-cf3c061042b5 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:11.558687 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 16/35] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:51:11.561989 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f455f-6716-4f61-9b03-0f765d404c9a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:11.562284 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f455f-6716-4f61-9b03-0f765d404c9a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:11.562532 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f455f-6716-4f61-9b03-0f765d404c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:11.601975 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-643f455f-6716-4f61-9b03-0f765d404c9a None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:11.603555 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-643f455f-6716-4f61-9b03-0f765d404c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:11.655458 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa14f4-e615-4677-9cd8-e60aba5b87c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:11.656128 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa14f4-e615-4677-9cd8-e60aba5b87c2 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:11.656457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa14f4-e615-4677-9cd8-e60aba5b87c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:11.656998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa14f4-e615-4677-9cd8-e60aba5b87c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:11.662827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:11.670566 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa14f4-e615-4677-9cd8-e60aba5b87c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:11.676569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-4afa14f4-e615-4677-9cd8-e60aba5b87c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:51:11.676569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:11.676569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:11.676569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:11.676569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:11.676569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:11.676569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:11.676569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:11.676569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:11.676569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:11.676569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:11.676569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:11.676569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:11.676569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:11.676569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:11.676569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:11.677501 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:11.677501 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:11.677501 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:11.677501 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:11.677501 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:11.677501 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:11.677501 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:11.677501 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:11.677501 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:11.677501 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:11.677501 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:11.677501 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:11.677501 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:11.677501 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:11.677501 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:11.677501 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:11.678252 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:11.678252 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:11.678252 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:11.678252 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:11.678252 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:11.678252 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:11.678252 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:11.678252 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:11.678252 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:11.678252 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:11.678252 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:51:11.678252 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:51:11.678252 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 20/36] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Oct 20 05:51:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:51:11.679084 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a9339-882b-4bd9-8cac-971acb2f9e02 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:11.679563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a9339-882b-4bd9-8cac-971acb2f9e02 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:11.679662 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a9339-882b-4bd9-8cac-971acb2f9e02 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:11.680964 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 21/37] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:11.685075 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a9339-882b-4bd9-8cac-971acb2f9e02 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:11.710066 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 17/38] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:11.710690 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49938a22-05d0-45a7-abd3-e5d4b67a2dd7 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:11.710990 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49938a22-05d0-45a7-abd3-e5d4b67a2dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:11.711336 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49938a22-05d0-45a7-abd3-e5d4b67a2dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:11.731688 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-49938a22-05d0-45a7-abd3-e5d4b67a2dd7 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:11.732560 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-49938a22-05d0-45a7-abd3-e5d4b67a2dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:11.742038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:11.819441 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a9339-882b-4bd9-8cac-971acb2f9e02 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:11.834580 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 18/39] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:11.858489 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da73e06-dca3-45a9-afce-3dc9c8ee03b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:11.859108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da73e06-dca3-45a9-afce-3dc9c8ee03b0 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:11.859389 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da73e06-dca3-45a9-afce-3dc9c8ee03b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:11.859662 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da73e06-dca3-45a9-afce-3dc9c8ee03b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:11.860325 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da73e06-dca3-45a9-afce-3dc9c8ee03b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:11.860634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da73e06-dca3-45a9-afce-3dc9c8ee03b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:11.862146 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da73e06-dca3-45a9-afce-3dc9c8ee03b0 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:11.871520 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 22/40] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:51:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:11.877104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49938a22-05d0-45a7-abd3-e5d4b67a2dd7 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:11.981881 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49938a22-05d0-45a7-abd3-e5d4b67a2dd7 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:11.984180 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:11.997904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da73e06-dca3-45a9-afce-3dc9c8ee03b0 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:12.022208 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91fd89-4212-428b-8463-37353c32ecb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:12.022851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91fd89-4212-428b-8463-37353c32ecb7 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:12.023129 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91fd89-4212-428b-8463-37353c32ecb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:12.023397 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91fd89-4212-428b-8463-37353c32ecb7 None admin] PATH_INFO: `/v3/users/6f317beb484644609d07d666527f760d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:12.024961 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d91fd89-4212-428b-8463-37353c32ecb7 None admin] RBAC: Authorizing `identity:get_user(user_id=6f317beb484644609d07d666527f760d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:12.026857 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d91fd89-4212-428b-8463-37353c32ecb7 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:12.032744 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 19/41] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:11 2022] GET /identity/v3/users/6f317beb484644609d07d666527f760d => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:12.033944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91fd89-4212-428b-8463-37353c32ecb7 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:12.113069 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da73e06-dca3-45a9-afce-3dc9c8ee03b0 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:12.114545 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:12.140536 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91fd89-4212-428b-8463-37353c32ecb7 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:12.149672 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf4c13-e4fe-4aca-812e-7f9ed552f48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:12.150649 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf4c13-e4fe-4aca-812e-7f9ed552f48b None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:12.151022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf4c13-e4fe-4aca-812e-7f9ed552f48b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:12.151312 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf4c13-e4fe-4aca-812e-7f9ed552f48b None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:12.151979 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf4c13-e4fe-4aca-812e-7f9ed552f48b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:12.153390 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf4c13-e4fe-4aca-812e-7f9ed552f48b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:12.167686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 23/42] 10.222.0.20 () {60 vars in 1131 bytes} [Thu Oct 20 05:51:11 2022] POST /identity/v3/roles => generated 223 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:51:12.169046 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf4c13-e4fe-4aca-812e-7f9ed552f48b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:12.176062 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd245479-0ebb-42e4-ba8b-fd0630def3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:12.177041 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd245479-0ebb-42e4-ba8b-fd0630def3b8 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:12.177336 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd245479-0ebb-42e4-ba8b-fd0630def3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:12.177612 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd245479-0ebb-42e4-ba8b-fd0630def3b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:12.178899 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd245479-0ebb-42e4-ba8b-fd0630def3b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:12.180599 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd245479-0ebb-42e4-ba8b-fd0630def3b8 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:12.182787 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 20/43] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:51:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:12.188818 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd245479-0ebb-42e4-ba8b-fd0630def3b8 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:12.238050 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:12.290402 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf4c13-e4fe-4aca-812e-7f9ed552f48b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:12.315827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd245479-0ebb-42e4-ba8b-fd0630def3b8 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:12.334799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e62756-6942-4f59-b0ee-15f696f256f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:12.335448 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e62756-6942-4f59-b0ee-15f696f256f4 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:12.335734 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e62756-6942-4f59-b0ee-15f696f256f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:12.336012 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e62756-6942-4f59-b0ee-15f696f256f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:12.337211 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e62756-6942-4f59-b0ee-15f696f256f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:12.338686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e62756-6942-4f59-b0ee-15f696f256f4 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:12.340279 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 24/44] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:51:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:12.346581 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e62756-6942-4f59-b0ee-15f696f256f4 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:12.353843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4fb84-d46e-41ec-af55-8ee5dab307b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:12.354464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4fb84-d46e-41ec-af55-8ee5dab307b0 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:12.354745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4fb84-d46e-41ec-af55-8ee5dab307b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:12.355021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4fb84-d46e-41ec-af55-8ee5dab307b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:12.355758 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f4fb84-d46e-41ec-af55-8ee5dab307b0 None admin] RBAC: Adding query filter params (role.id=d8debbb193834fac9cc25f81726f3a67, scope.domain.id=default, user.id=6f317beb484644609d07d666527f760d) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:12.356073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f4fb84-d46e-41ec-af55-8ee5dab307b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:12.357569 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f4fb84-d46e-41ec-af55-8ee5dab307b0 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:12.369219 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 21/45] 10.222.0.20 () {56 vars in 1322 bytes} [Thu Oct 20 05:51:12 2022] GET /identity/v3/role_assignments?user.id=6f317beb484644609d07d666527f760d&scope.domain.id=default&role.id=d8debbb193834fac9cc25f81726f3a67 => generated 530 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:12.381966 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:12.462803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e62756-6942-4f59-b0ee-15f696f256f4 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:12.498543 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33180b37-5ee7-4ddc-b3ce-ded984f30ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:12.499162 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33180b37-5ee7-4ddc-b3ce-ded984f30ed8 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:12.499446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33180b37-5ee7-4ddc-b3ce-ded984f30ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:12.499724 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33180b37-5ee7-4ddc-b3ce-ded984f30ed8 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:12.500395 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33180b37-5ee7-4ddc-b3ce-ded984f30ed8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:12.501896 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33180b37-5ee7-4ddc-b3ce-ded984f30ed8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:12.534306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 25/46] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Oct 20 05:51:12 2022] POST /identity/v3/projects => generated 312 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:51:14.703600 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff6b2a-35a1-48e0-a64b-a7dbc9b33d69 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:14.704109 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff6b2a-35a1-48e0-a64b-a7dbc9b33d69 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:14.704194 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff6b2a-35a1-48e0-a64b-a7dbc9b33d69 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:14.705518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 22/47] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:14.710915 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f15a93-8741-4ca5-bf0a-e2471401a02b None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:14.711340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f15a93-8741-4ca5-bf0a-e2471401a02b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:14.711470 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f15a93-8741-4ca5-bf0a-e2471401a02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:14.729251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-98f15a93-8741-4ca5-bf0a-e2471401a02b None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:14.730468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-98f15a93-8741-4ca5-bf0a-e2471401a02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:14.794563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 26/48] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:15.064209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff6b2a-35a1-48e0-a64b-a7dbc9b33d69 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:15.162843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff6b2a-35a1-48e0-a64b-a7dbc9b33d69 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:15.165131 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:15.196650 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec115c4-2525-48f5-80f9-1e35eb63929e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:15.197312 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec115c4-2525-48f5-80f9-1e35eb63929e None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:15.197593 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec115c4-2525-48f5-80f9-1e35eb63929e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:15.197867 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec115c4-2525-48f5-80f9-1e35eb63929e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:15.198972 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec115c4-2525-48f5-80f9-1e35eb63929e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:15.200423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec115c4-2525-48f5-80f9-1e35eb63929e None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:15.201995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 23/49] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:51:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:15.207454 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f15a93-8741-4ca5-bf0a-e2471401a02b None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:15.220995 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:15.304764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f15a93-8741-4ca5-bf0a-e2471401a02b None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:15.337786 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23760d-c5d6-46b9-88f7-9ea8725929ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:15.338327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23760d-c5d6-46b9-88f7-9ea8725929ed None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:15.338605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23760d-c5d6-46b9-88f7-9ea8725929ed None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:15.338877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23760d-c5d6-46b9-88f7-9ea8725929ed None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:15.339516 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23760d-c5d6-46b9-88f7-9ea8725929ed None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:15.341003 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23760d-c5d6-46b9-88f7-9ea8725929ed None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:15.367225 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 27/50] 10.222.0.20 () {60 vars in 1140 bytes} [Thu Oct 20 05:51:15 2022] POST /identity/v3/projects => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:51:17.487941 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc494471-0f94-4848-a339-28e3e77316f2 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:17.488571 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc494471-0f94-4848-a339-28e3e77316f2 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:17.488571 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc494471-0f94-4848-a339-28e3e77316f2 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:17.489836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 24/51] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:17.495041 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6596adf-2961-45d0-b46e-253850f403c7 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:17.495403 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6596adf-2961-45d0-b46e-253850f403c7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:17.495584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6596adf-2961-45d0-b46e-253850f403c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:17.514023 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-c6596adf-2961-45d0-b46e-253850f403c7 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:17.515188 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c6596adf-2961-45d0-b46e-253850f403c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:17.570778 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:17.579832 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 28/52] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:17.854954 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc494471-0f94-4848-a339-28e3e77316f2 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:17.953156 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc494471-0f94-4848-a339-28e3e77316f2 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:17.986370 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159c8c9-692d-4718-9da1-465aac106ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:17.986990 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159c8c9-692d-4718-9da1-465aac106ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:17.987273 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159c8c9-692d-4718-9da1-465aac106ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:17.987541 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159c8c9-692d-4718-9da1-465aac106ce6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:17.989105 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d159c8c9-692d-4718-9da1-465aac106ce6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:17.990533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d159c8c9-692d-4718-9da1-465aac106ce6 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:17.992068 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 25/53] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:51:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:17.997687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6596adf-2961-45d0-b46e-253850f403c7 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:18.093235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6596adf-2961-45d0-b46e-253850f403c7 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:18.125492 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44568924-fc92-4c73-9ecc-90a840df4c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:18.126040 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44568924-fc92-4c73-9ecc-90a840df4c4b None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:18.126321 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44568924-fc92-4c73-9ecc-90a840df4c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:18.126595 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44568924-fc92-4c73-9ecc-90a840df4c4b None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:18.127243 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44568924-fc92-4c73-9ecc-90a840df4c4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:18.129403 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44568924-fc92-4c73-9ecc-90a840df4c4b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:18.156298 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 29/54] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Oct 20 05:51:17 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:51:21.511289 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709dceba-aeb8-40cb-819a-5d336882164c None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:21.511289 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709dceba-aeb8-40cb-819a-5d336882164c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:21.511289 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709dceba-aeb8-40cb-819a-5d336882164c None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:21.511289 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 26/55] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:21.516727 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ec82c-d6f7-4cae-b36d-726e22bc5ab7 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:21.517530 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ec82c-d6f7-4cae-b36d-726e22bc5ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:21.518011 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ec82c-d6f7-4cae-b36d-726e22bc5ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:21.544733 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-9d3ec82c-d6f7-4cae-b36d-726e22bc5ab7 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:21.546212 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9d3ec82c-d6f7-4cae-b36d-726e22bc5ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:21.559338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c48a3-c28c-46e3-a946-c61964da06c2 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:21.559587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c48a3-c28c-46e3-a946-c61964da06c2 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:21.559809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c48a3-c28c-46e3-a946-c61964da06c2 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:21.561448 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 27/56] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:51:21.562386 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77274f-0878-48dc-b887-6134c14a5116 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:21.562742 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77274f-0878-48dc-b887-6134c14a5116 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:21.562991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77274f-0878-48dc-b887-6134c14a5116 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:21.564160 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 28/57] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:21.569674 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539769a0-cb20-4590-8c68-ed749da9477a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:21.570201 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539769a0-cb20-4590-8c68-ed749da9477a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:21.570660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539769a0-cb20-4590-8c68-ed749da9477a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:21.591186 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-539769a0-cb20-4590-8c68-ed749da9477a None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:21.592737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-539769a0-cb20-4590-8c68-ed749da9477a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:21.607435 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:21.632567 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 30/58] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 05:51:21.634397 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407bb2c-59ff-48b0-a4cb-ab7f315afc12 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:21.634819 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407bb2c-59ff-48b0-a4cb-ab7f315afc12 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:21.635225 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407bb2c-59ff-48b0-a4cb-ab7f315afc12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:21.642555 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:21.656012 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-7407bb2c-59ff-48b0-a4cb-ab7f315afc12 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:21.657603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7407bb2c-59ff-48b0-a4cb-ab7f315afc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:21.690762 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 29/59] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 05:51:21.764227 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 31/60] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:21.804639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43233f60-f792-48a6-802d-8b3d01a5b9ac None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:21.805202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43233f60-f792-48a6-802d-8b3d01a5b9ac None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:21.805615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43233f60-f792-48a6-802d-8b3d01a5b9ac None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:21.807116 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 30/61] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:21.812531 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d327711-32c1-41f2-b671-b40b5128275b None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:21.813002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d327711-32c1-41f2-b671-b40b5128275b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:21.813429 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d327711-32c1-41f2-b671-b40b5128275b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:21.833339 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-8d327711-32c1-41f2-b671-b40b5128275b None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:21.834695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8d327711-32c1-41f2-b671-b40b5128275b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:21.943940 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 32/62] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:21.954005 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43233f60-f792-48a6-802d-8b3d01a5b9ac None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:22.014575 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d327711-32c1-41f2-b671-b40b5128275b None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:22.063540 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:22.131827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d327711-32c1-41f2-b671-b40b5128275b None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:22.134030 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43233f60-f792-48a6-802d-8b3d01a5b9ac None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:22.169504 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be8ebf-7adb-419a-8c49-4c3cd4422a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:22.170419 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be8ebf-7adb-419a-8c49-4c3cd4422a18 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:22.170843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be8ebf-7adb-419a-8c49-4c3cd4422a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:22.170953 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab13864-0444-4fae-8ae0-e9ac33acc542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:22.171375 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be8ebf-7adb-419a-8c49-4c3cd4422a18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:22.171537 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab13864-0444-4fae-8ae0-e9ac33acc542 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:22.172003 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab13864-0444-4fae-8ae0-e9ac33acc542 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:22.172277 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab13864-0444-4fae-8ae0-e9ac33acc542 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:22.183246 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56be8ebf-7adb-419a-8c49-4c3cd4422a18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:22.185409 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab13864-0444-4fae-8ae0-e9ac33acc542 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:22.188292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-56be8ebf-7adb-419a-8c49-4c3cd4422a18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:51:22.188292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:22.188292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:22.188292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:22.188292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:22.188292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:22.188292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:22.188292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:22.188292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:22.188292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:22.188292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:22.188292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:22.188292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:22.188292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:22.188292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:22.188292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:22.189349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:22.189349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:22.189349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:22.189349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:22.189349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:22.189349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:22.189349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:22.189349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:22.189349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:22.189349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:22.189349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:22.189349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:22.189349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:22.189349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:22.189349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:22.189349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:22.190306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:22.190306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:22.190306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:22.190306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:22.190306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:22.190306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:22.190306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:22.190306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:22.190306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:22.190306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:22.190306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:51:22.190306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:51:22.190306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 33/63] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Oct 20 05:51:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:51:22.190306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be8ebf-7adb-419a-8c49-4c3cd4422a18 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:22.196886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-6ab13864-0444-4fae-8ae0-e9ac33acc542 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:51:22.196886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:22.196886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:22.196886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:22.196886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:22.196886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:22.196886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:22.196886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:22.196886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:22.196886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:22.196886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:22.196886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:22.196886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:22.196886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:22.196886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:22.196886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:22.197775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:22.197775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:22.197775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:22.197775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:22.197775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:22.197775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:22.197775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:22.197775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:22.197775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:22.197775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:22.197775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:22.197775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:22.197775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:22.197775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:22.197775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:22.197775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:22.198538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:22.198538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:22.198538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:22.198538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:22.198538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:22.198538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:22.198538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:22.198538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:22.198538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:22.198538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:22.198538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:51:22.198538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:51:22.202564 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 31/64] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:51:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:51:22.202564 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab13864-0444-4fae-8ae0-e9ac33acc542 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:22.239616 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:22.305691 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:22.308865 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be8ebf-7adb-419a-8c49-4c3cd4422a18 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:22.317804 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab13864-0444-4fae-8ae0-e9ac33acc542 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:22.347055 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8fda9-eb1b-4b70-b3e8-f4b94e1e4bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:22.348391 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8fda9-eb1b-4b70-b3e8-f4b94e1e4bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:22.348689 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8fda9-eb1b-4b70-b3e8-f4b94e1e4bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:22.349013 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8fda9-eb1b-4b70-b3e8-f4b94e1e4bb2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:22.357190 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63b174-44ff-4d62-8e0e-f405b457d84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:22.357819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63b174-44ff-4d62-8e0e-f405b457d84a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:22.358120 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63b174-44ff-4d62-8e0e-f405b457d84a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:22.358398 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63b174-44ff-4d62-8e0e-f405b457d84a None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:22.359071 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63b174-44ff-4d62-8e0e-f405b457d84a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:22.359378 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63b174-44ff-4d62-8e0e-f405b457d84a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:22.360885 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63b174-44ff-4d62-8e0e-f405b457d84a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:22.361406 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8fda9-eb1b-4b70-b3e8-f4b94e1e4bb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:22.366483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-8ee8fda9-eb1b-4b70-b3e8-f4b94e1e4bb2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 05:51:22.366483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:22.366483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:22.366483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:22.366483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:22.366483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:22.366483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:22.366483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:22.366483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:22.366483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:22.366483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:22.366483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:22.366483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:22.366483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:22.366483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:22.366483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:22.367286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:22.367286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:22.367286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:22.367286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:22.367286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:22.367286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:22.367286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:22.367286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:22.367286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:22.367286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:22.367286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:22.367286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:22.367286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:22.367286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:22.367286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:22.367286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:22.368034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:22.368034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:22.368034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:22.368034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:22.368034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:22.368034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:22.368034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:22.368034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:22.368034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:22.368034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:22.368034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 05:51:22.368034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:51:22.368034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 34/65] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Oct 20 05:51:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:51:22.369291 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8fda9-eb1b-4b70-b3e8-f4b94e1e4bb2 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:22.379551 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 32/66] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:51:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:22.385010 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63b174-44ff-4d62-8e0e-f405b457d84a None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:22.478201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8fda9-eb1b-4b70-b3e8-f4b94e1e4bb2 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:22.515105 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315663b1-7812-48d5-b7e3-e11dd2e73ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:22.515734 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315663b1-7812-48d5-b7e3-e11dd2e73ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:22.516017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315663b1-7812-48d5-b7e3-e11dd2e73ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:22.516293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315663b1-7812-48d5-b7e3-e11dd2e73ad2 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:22.517021 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315663b1-7812-48d5-b7e3-e11dd2e73ad2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:22.517557 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315663b1-7812-48d5-b7e3-e11dd2e73ad2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:22.518731 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315663b1-7812-48d5-b7e3-e11dd2e73ad2 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:22.527727 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 35/67] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Oct 20 05:51:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:22.528580 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315663b1-7812-48d5-b7e3-e11dd2e73ad2 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:22.555282 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:22.587969 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:22.596838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63b174-44ff-4d62-8e0e-f405b457d84a None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:22.629420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315663b1-7812-48d5-b7e3-e11dd2e73ad2 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:22.631441 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3235334-5606-4318-a25c-877d0b0fd097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:22.632085 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3235334-5606-4318-a25c-877d0b0fd097 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:22.632367 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3235334-5606-4318-a25c-877d0b0fd097 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:22.632646 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3235334-5606-4318-a25c-877d0b0fd097 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:22.643576 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3235334-5606-4318-a25c-877d0b0fd097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:22.648753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-d3235334-5606-4318-a25c-877d0b0fd097 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:51:22.648753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:22.648753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:22.648753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:22.648753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:22.648753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:22.648753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:22.648753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:22.648753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:22.648753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:22.648753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:22.648753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:22.648753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:22.648753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:22.648753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:22.648753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:22.649774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:22.649774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:22.649774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:22.649774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:22.649774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:22.649774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:22.649774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:22.649774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:22.649774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:22.649774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:22.649774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:22.649774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:22.649774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:22.649774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:22.649774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:22.649774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:22.650542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:22.650542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:22.650542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:22.650542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:22.650542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:22.650542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:22.650542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:22.650542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:22.650542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:22.650542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:22.650542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:51:22.650542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:51:22.650542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 33/68] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:51:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:51:22.651175 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3235334-5606-4318-a25c-877d0b0fd097 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:22.667008 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b203f1-fe68-4cdb-9054-accb7d95803f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:22.667633 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b203f1-fe68-4cdb-9054-accb7d95803f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:22.667916 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b203f1-fe68-4cdb-9054-accb7d95803f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:22.668202 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b203f1-fe68-4cdb-9054-accb7d95803f None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:22.668891 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b203f1-fe68-4cdb-9054-accb7d95803f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:22.669198 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b203f1-fe68-4cdb-9054-accb7d95803f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:22.670582 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b203f1-fe68-4cdb-9054-accb7d95803f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:22.678653 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 36/69] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:51:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:22.683222 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fb524-f511-452d-b3bb-f90549059474 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:22.683222 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fb524-f511-452d-b3bb-f90549059474 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:22.683222 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fb524-f511-452d-b3bb-f90549059474 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:22.683222 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 37/70] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:22.683222 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fb524-f511-452d-b3bb-f90549059474 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:22.757701 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3235334-5606-4318-a25c-877d0b0fd097 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:22.768375 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:22.785697 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fb524-f511-452d-b3bb-f90549059474 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:22.792349 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159e173-6220-4640-bec5-372f54aa3542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:22.793021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159e173-6220-4640-bec5-372f54aa3542 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:22.793304 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159e173-6220-4640-bec5-372f54aa3542 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:22.793581 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159e173-6220-4640-bec5-372f54aa3542 None admin] PATH_INFO: `/v3/users/6f317beb484644609d07d666527f760d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:22.794724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159e173-6220-4640-bec5-372f54aa3542 None admin] RBAC: Authorizing `identity:get_user(user_id=6f317beb484644609d07d666527f760d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:22.796177 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159e173-6220-4640-bec5-372f54aa3542 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:22.797790 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 34/71] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:22 2022] GET /identity/v3/users/6f317beb484644609d07d666527f760d => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:22.802535 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159e173-6220-4640-bec5-372f54aa3542 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:22.819458 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729aeba-f696-4dbc-9e13-52f47e2ad0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:22.820088 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729aeba-f696-4dbc-9e13-52f47e2ad0bc None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:22.820371 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729aeba-f696-4dbc-9e13-52f47e2ad0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:22.820649 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729aeba-f696-4dbc-9e13-52f47e2ad0bc None admin] PATH_INFO: `/v3/users/3ca0dc0787674530a74f83cfa41df3d6` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:22.821990 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:22.834527 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c729aeba-f696-4dbc-9e13-52f47e2ad0bc None admin] RBAC: Authorizing `identity:get_user(user_id=3ca0dc0787674530a74f83cfa41df3d6)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:22.835973 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c729aeba-f696-4dbc-9e13-52f47e2ad0bc None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:22.837616 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 38/72] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:22 2022] GET /identity/v3/users/3ca0dc0787674530a74f83cfa41df3d6 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:22.838802 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e0858-54f9-4c0d-9ff2-a60aca8047c8 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:22.839032 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e0858-54f9-4c0d-9ff2-a60aca8047c8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:22.839306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e0858-54f9-4c0d-9ff2-a60aca8047c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:22.856744 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-ee0e0858-54f9-4c0d-9ff2-a60aca8047c8 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:22.857944 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ee0e0858-54f9-4c0d-9ff2-a60aca8047c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:22.900778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159e173-6220-4640-bec5-372f54aa3542 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:22.923354 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 39/73] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:22.924199 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e0858-54f9-4c0d-9ff2-a60aca8047c8 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:22.940251 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1078be-455d-4c48-aeec-7a1dafe8952d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:22.941150 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1078be-455d-4c48-aeec-7a1dafe8952d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:22.942079 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1078be-455d-4c48-aeec-7a1dafe8952d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:22.942511 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1078be-455d-4c48-aeec-7a1dafe8952d None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:22.943710 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1078be-455d-4c48-aeec-7a1dafe8952d None admin] RBAC: Adding query filter params (name=member) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:22.944166 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1078be-455d-4c48-aeec-7a1dafe8952d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:22.946240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1078be-455d-4c48-aeec-7a1dafe8952d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:22.957617 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 35/74] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Oct 20 05:51:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:22.959686 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1078be-455d-4c48-aeec-7a1dafe8952d None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:22.995584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:23.027046 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e0858-54f9-4c0d-9ff2-a60aca8047c8 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:23.053596 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:23.064053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f90b3-e108-49a4-afc4-fed8700b79e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:23.064713 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f90b3-e108-49a4-afc4-fed8700b79e8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:23.064999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f90b3-e108-49a4-afc4-fed8700b79e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:23.065273 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f90b3-e108-49a4-afc4-fed8700b79e8 None admin] PATH_INFO: `/v3/users/3ca0dc0787674530a74f83cfa41df3d6` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:23.066995 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f90b3-e108-49a4-afc4-fed8700b79e8 None admin] RBAC: Authorizing `identity:get_user(user_id=3ca0dc0787674530a74f83cfa41df3d6)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:23.068433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f90b3-e108-49a4-afc4-fed8700b79e8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:23.072737 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 40/75] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:22 2022] GET /identity/v3/users/3ca0dc0787674530a74f83cfa41df3d6 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:23.072737 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f90b3-e108-49a4-afc4-fed8700b79e8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:23.079809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1078be-455d-4c48-aeec-7a1dafe8952d None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:23.117100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d40d47-b15b-40e5-adbf-8df083e26f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:23.118048 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d40d47-b15b-40e5-adbf-8df083e26f6f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:23.118520 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d40d47-b15b-40e5-adbf-8df083e26f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:23.118967 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d40d47-b15b-40e5-adbf-8df083e26f6f None admin] PATH_INFO: `/v3/projects/e78108de05fe409aa6e56461abb56cc0` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:23.120289 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d40d47-b15b-40e5-adbf-8df083e26f6f None admin] RBAC: Authorizing `identity:get_project(project_id=e78108de05fe409aa6e56461abb56cc0)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:23.122055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d40d47-b15b-40e5-adbf-8df083e26f6f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:23.123929 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 36/76] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:51:22 2022] GET /identity/v3/projects/e78108de05fe409aa6e56461abb56cc0 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:23.124884 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d40d47-b15b-40e5-adbf-8df083e26f6f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:23.174497 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f90b3-e108-49a4-afc4-fed8700b79e8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:23.208711 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945cb71-8961-4ea4-95d7-fdc53f6509df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:23.209348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945cb71-8961-4ea4-95d7-fdc53f6509df None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:23.209628 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945cb71-8961-4ea4-95d7-fdc53f6509df None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:23.209900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945cb71-8961-4ea4-95d7-fdc53f6509df None admin] PATH_INFO: `/v3/projects/e78108de05fe409aa6e56461abb56cc0` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:23.211048 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945cb71-8961-4ea4-95d7-fdc53f6509df None admin] RBAC: Authorizing `identity:get_project(project_id=e78108de05fe409aa6e56461abb56cc0)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:23.212479 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945cb71-8961-4ea4-95d7-fdc53f6509df None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:23.214153 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 41/77] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:51:23 2022] GET /identity/v3/projects/e78108de05fe409aa6e56461abb56cc0 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:23.214907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945cb71-8961-4ea4-95d7-fdc53f6509df None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:23.226012 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:23.232987 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d40d47-b15b-40e5-adbf-8df083e26f6f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:23.266890 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee8efe-9afb-4a41-a784-d7e7759fd1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:23.267524 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee8efe-9afb-4a41-a784-d7e7759fd1ec None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:23.267809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee8efe-9afb-4a41-a784-d7e7759fd1ec None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:23.268091 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee8efe-9afb-4a41-a784-d7e7759fd1ec None admin] PATH_INFO: `/v3/users/3ca0dc0787674530a74f83cfa41df3d6` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:23.269867 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee8efe-9afb-4a41-a784-d7e7759fd1ec None admin] RBAC: Authorizing `identity:get_user(user_id=3ca0dc0787674530a74f83cfa41df3d6)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:23.271522 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee8efe-9afb-4a41-a784-d7e7759fd1ec None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:23.273136 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 37/78] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:23 2022] GET /identity/v3/users/3ca0dc0787674530a74f83cfa41df3d6 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:23.277871 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee8efe-9afb-4a41-a784-d7e7759fd1ec None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:23.296629 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:23.314544 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945cb71-8961-4ea4-95d7-fdc53f6509df None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:23.347961 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272549f-0d79-4a43-afd1-9e8bf771b0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:23.348591 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272549f-0d79-4a43-afd1-9e8bf771b0ae None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:23.348905 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272549f-0d79-4a43-afd1-9e8bf771b0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:23.349178 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272549f-0d79-4a43-afd1-9e8bf771b0ae None admin] PATH_INFO: `/v3/projects/e78108de05fe409aa6e56461abb56cc0` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:23.350289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272549f-0d79-4a43-afd1-9e8bf771b0ae None admin] RBAC: Authorizing `identity:get_project(project_id=e78108de05fe409aa6e56461abb56cc0)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:23.351725 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272549f-0d79-4a43-afd1-9e8bf771b0ae None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:23.353386 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 42/79] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:51:23 2022] GET /identity/v3/projects/e78108de05fe409aa6e56461abb56cc0 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:23.354158 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272549f-0d79-4a43-afd1-9e8bf771b0ae None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:23.379545 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee8efe-9afb-4a41-a784-d7e7759fd1ec None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:23.413034 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50442a08-ee93-4370-b586-9da01e8ca8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:23.413689 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50442a08-ee93-4370-b586-9da01e8ca8c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:23.413966 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50442a08-ee93-4370-b586-9da01e8ca8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:23.414239 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50442a08-ee93-4370-b586-9da01e8ca8c9 None admin] PATH_INFO: `/v3/projects/e78108de05fe409aa6e56461abb56cc0/users/6f317beb484644609d07d666527f760d/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:23.416150 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50442a08-ee93-4370-b586-9da01e8ca8c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=e78108de05fe409aa6e56461abb56cc0, user_id=6f317beb484644609d07d666527f760d, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:23.417630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50442a08-ee93-4370-b586-9da01e8ca8c9 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:23.431536 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 38/80] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:51:23 2022] PUT /identity/v3/projects/e78108de05fe409aa6e56461abb56cc0/users/6f317beb484644609d07d666527f760d/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:51:23.432316 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50442a08-ee93-4370-b586-9da01e8ca8c9 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:23.439075 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:23.467476 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272549f-0d79-4a43-afd1-9e8bf771b0ae None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:23.506139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a94b0-0cb2-4513-b199-3f461b9bffe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:23.507199 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a94b0-0cb2-4513-b199-3f461b9bffe1 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:23.507644 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a94b0-0cb2-4513-b199-3f461b9bffe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:23.507930 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a94b0-0cb2-4513-b199-3f461b9bffe1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:23.509428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a94b0-0cb2-4513-b199-3f461b9bffe1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:23.510918 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a94b0-0cb2-4513-b199-3f461b9bffe1 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:23.512576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 43/81] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:51:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:23.517707 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a94b0-0cb2-4513-b199-3f461b9bffe1 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:23.529994 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:23.558167 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50442a08-ee93-4370-b586-9da01e8ca8c9 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:23.597420 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059dc22-f565-4dd6-be70-ae79874e8d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:23.598253 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059dc22-f565-4dd6-be70-ae79874e8d14 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:23.598517 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059dc22-f565-4dd6-be70-ae79874e8d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:23.598676 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059dc22-f565-4dd6-be70-ae79874e8d14 None admin] PATH_INFO: `/v3/projects/e78108de05fe409aa6e56461abb56cc0/users/3ca0dc0787674530a74f83cfa41df3d6/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:23.600627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059dc22-f565-4dd6-be70-ae79874e8d14 None admin] RBAC: Authorizing `identity:create_grant(project_id=e78108de05fe409aa6e56461abb56cc0, user_id=3ca0dc0787674530a74f83cfa41df3d6, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:23.602127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059dc22-f565-4dd6-be70-ae79874e8d14 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:23.618255 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 39/82] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:51:23 2022] PUT /identity/v3/projects/e78108de05fe409aa6e56461abb56cc0/users/3ca0dc0787674530a74f83cfa41df3d6/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:51:23.618736 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059dc22-f565-4dd6-be70-ae79874e8d14 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:23.629770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a94b0-0cb2-4513-b199-3f461b9bffe1 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:23.670257 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c9357-983e-4204-800b-95cdb1bb8e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:23.671363 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c9357-983e-4204-800b-95cdb1bb8e38 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:23.671846 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c9357-983e-4204-800b-95cdb1bb8e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:23.672319 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c9357-983e-4204-800b-95cdb1bb8e38 None admin] PATH_INFO: `/v3/projects/883ca9f8306a4bcdaa646fb2f2b656a2` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:23.673831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c9357-983e-4204-800b-95cdb1bb8e38 None admin] RBAC: Authorizing `identity:get_project(project_id=883ca9f8306a4bcdaa646fb2f2b656a2)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:23.676267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c9357-983e-4204-800b-95cdb1bb8e38 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:23.679564 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 44/83] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:51:23 2022] GET /identity/v3/projects/883ca9f8306a4bcdaa646fb2f2b656a2 => generated 312 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:23.680675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c9357-983e-4204-800b-95cdb1bb8e38 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:23.701394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:23.777741 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059dc22-f565-4dd6-be70-ae79874e8d14 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:23.812380 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c9357-983e-4204-800b-95cdb1bb8e38 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:23.818568 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69945424-3df4-4e24-bf12-53fda02bef0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:23.819310 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69945424-3df4-4e24-bf12-53fda02bef0a None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:23.819592 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69945424-3df4-4e24-bf12-53fda02bef0a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:23.819867 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69945424-3df4-4e24-bf12-53fda02bef0a None admin] PATH_INFO: `/v3/projects/e78108de05fe409aa6e56461abb56cc0/users/3ca0dc0787674530a74f83cfa41df3d6/roles/49dae0405c764c96b7bfbcf9d9efd011` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:23.824178 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69945424-3df4-4e24-bf12-53fda02bef0a None admin] RBAC: Authorizing `identity:create_grant(project_id=e78108de05fe409aa6e56461abb56cc0, user_id=3ca0dc0787674530a74f83cfa41df3d6, role_id=49dae0405c764c96b7bfbcf9d9efd011)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:23.825656 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69945424-3df4-4e24-bf12-53fda02bef0a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:23.842358 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:23.851058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 40/84] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:51:23 2022] PUT /identity/v3/projects/e78108de05fe409aa6e56461abb56cc0/users/3ca0dc0787674530a74f83cfa41df3d6/roles/49dae0405c764c96b7bfbcf9d9efd011 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:51:23.851721 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69945424-3df4-4e24-bf12-53fda02bef0a None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:23.852599 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2fa7f-b17b-4a8f-9411-96ae1b0dee94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:23.853323 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2fa7f-b17b-4a8f-9411-96ae1b0dee94 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:23.853605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2fa7f-b17b-4a8f-9411-96ae1b0dee94 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:23.853882 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2fa7f-b17b-4a8f-9411-96ae1b0dee94 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:23.854544 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa2fa7f-b17b-4a8f-9411-96ae1b0dee94 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:23.856026 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa2fa7f-b17b-4a8f-9411-96ae1b0dee94 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:23.894579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 45/85] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Oct 20 05:51:23 2022] POST /identity/v3/users => generated 289 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:51:24.037004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69945424-3df4-4e24-bf12-53fda02bef0a None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:24.086458 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da00dfb-d938-441b-9d3a-c7c9fa909cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:24.087148 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da00dfb-d938-441b-9d3a-c7c9fa909cb7 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:24.087583 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da00dfb-d938-441b-9d3a-c7c9fa909cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:24.087866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da00dfb-d938-441b-9d3a-c7c9fa909cb7 None admin] PATH_INFO: `/v3/projects/883ca9f8306a4bcdaa646fb2f2b656a2/users/3ca0dc0787674530a74f83cfa41df3d6/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:24.089964 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da00dfb-d938-441b-9d3a-c7c9fa909cb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=883ca9f8306a4bcdaa646fb2f2b656a2, user_id=3ca0dc0787674530a74f83cfa41df3d6, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:24.091479 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da00dfb-d938-441b-9d3a-c7c9fa909cb7 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:24.107939 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 41/86] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:51:23 2022] PUT /identity/v3/projects/883ca9f8306a4bcdaa646fb2f2b656a2/users/3ca0dc0787674530a74f83cfa41df3d6/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:51:26.106695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de102d-94c7-4dcc-a7e8-966cb057b7bc None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:26.107212 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de102d-94c7-4dcc-a7e8-966cb057b7bc None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:26.107320 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de102d-94c7-4dcc-a7e8-966cb057b7bc None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:26.108862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 46/87] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:26.126605 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3dcebe-7d5b-4994-8c29-e133fe5d0c85 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:26.127063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3dcebe-7d5b-4994-8c29-e133fe5d0c85 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:26.127249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3dcebe-7d5b-4994-8c29-e133fe5d0c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:26.153648 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-8b3dcebe-7d5b-4994-8c29-e133fe5d0c85 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:26.154840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8b3dcebe-7d5b-4994-8c29-e133fe5d0c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:26.219176 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:26.239729 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 42/88] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:26.607193 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46665432-d8d4-4f13-8655-2c443ef5edae None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:26.607561 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46665432-d8d4-4f13-8655-2c443ef5edae None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:26.607755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46665432-d8d4-4f13-8655-2c443ef5edae None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:26.609259 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 47/89] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:26.614402 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd506df5-a92d-4b9d-bd3d-954b24d59de5 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:26.614691 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd506df5-a92d-4b9d-bd3d-954b24d59de5 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:26.614969 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd506df5-a92d-4b9d-bd3d-954b24d59de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:26.639355 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-dd506df5-a92d-4b9d-bd3d-954b24d59de5 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:26.640534 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-dd506df5-a92d-4b9d-bd3d-954b24d59de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:26.716291 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46665432-d8d4-4f13-8655-2c443ef5edae None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:26.723068 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 43/90] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:26.822345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:26.878420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46665432-d8d4-4f13-8655-2c443ef5edae None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:26.925619 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20419f69-8b76-4e94-b6a1-f00f032a284d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:26.926698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20419f69-8b76-4e94-b6a1-f00f032a284d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:26.927179 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20419f69-8b76-4e94-b6a1-f00f032a284d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:26.927642 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20419f69-8b76-4e94-b6a1-f00f032a284d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:26.938879 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20419f69-8b76-4e94-b6a1-f00f032a284d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:26.944990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-20419f69-8b76-4e94-b6a1-f00f032a284d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:51:26.944990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:26.944990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:26.944990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:26.944990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:26.944990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:26.944990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:26.944990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:26.944990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:26.944990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:26.944990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:26.944990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:26.944990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:26.944990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:26.944990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:26.944990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:26.945840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:26.945840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:26.945840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:26.945840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:26.945840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:26.945840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:26.945840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:26.945840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:26.945840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:26.945840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:26.945840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:26.945840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:26.945840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:26.945840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:26.945840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:26.945840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:26.946699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:26.946699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:26.946699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:26.946699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:26.946699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:26.946699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:26.946699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:26.946699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:26.946699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:26.946699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:26.946699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:51:26.946699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:51:26.948923 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 48/91] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Oct 20 05:51:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:51:26.958572 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd506df5-a92d-4b9d-bd3d-954b24d59de5 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:27.018524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20419f69-8b76-4e94-b6a1-f00f032a284d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:27.126007 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd506df5-a92d-4b9d-bd3d-954b24d59de5 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:27.164571 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:27.198409 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20419f69-8b76-4e94-b6a1-f00f032a284d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:27.212368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24abe204-c14c-40d5-8c9a-aed7bdb2f033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:27.213396 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24abe204-c14c-40d5-8c9a-aed7bdb2f033 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:27.213907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24abe204-c14c-40d5-8c9a-aed7bdb2f033 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:27.214727 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24abe204-c14c-40d5-8c9a-aed7bdb2f033 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:27.215617 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24abe204-c14c-40d5-8c9a-aed7bdb2f033 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:27.216118 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24abe204-c14c-40d5-8c9a-aed7bdb2f033 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:27.217760 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24abe204-c14c-40d5-8c9a-aed7bdb2f033 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:27.236180 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 44/92] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:51:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:27.242693 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24abe204-c14c-40d5-8c9a-aed7bdb2f033 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:27.255745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:27.269529 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8c0e0-e241-42b1-8ecc-f256a729eb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:27.272278 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8c0e0-e241-42b1-8ecc-f256a729eb37 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:27.274278 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8c0e0-e241-42b1-8ecc-f256a729eb37 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:27.274847 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8c0e0-e241-42b1-8ecc-f256a729eb37 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:27.293629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d8c0e0-e241-42b1-8ecc-f256a729eb37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:27.309297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-18d8c0e0-e241-42b1-8ecc-f256a729eb37 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:51:27.309297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:27.309297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:27.309297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:27.309297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:27.309297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:27.309297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:27.309297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:27.309297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:27.309297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:27.309297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:27.309297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:27.309297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:27.309297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:27.309297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:27.309297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:27.310192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:27.310192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:27.310192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:27.310192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:27.310192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:27.310192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:27.310192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:27.310192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:27.310192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:27.310192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:27.310192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:27.310192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:27.310192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:27.310192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:27.310192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:27.310192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:27.311028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:27.311028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:27.311028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:27.311028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:27.311028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:27.311028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:27.311028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:27.311028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:27.311028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:27.311028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:27.311028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:51:27.311028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:51:27.311028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 49/93] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:51:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:51:27.312172 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55da55-6a7c-40f2-898f-f050d60d0035 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:27.312172 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55da55-6a7c-40f2-898f-f050d60d0035 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:27.312317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55da55-6a7c-40f2-898f-f050d60d0035 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:27.316755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 50/94] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:27.316755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f9bc5-ad5b-48cf-b704-c94e64adc8e8 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:27.316755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f9bc5-ad5b-48cf-b704-c94e64adc8e8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:27.316755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f9bc5-ad5b-48cf-b704-c94e64adc8e8 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:27.317246 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 51/95] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:27.322696 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966823e3-e9db-4a51-a2c2-b471fbcc5585 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:27.322696 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966823e3-e9db-4a51-a2c2-b471fbcc5585 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:27.322899 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966823e3-e9db-4a51-a2c2-b471fbcc5585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:27.341595 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-966823e3-e9db-4a51-a2c2-b471fbcc5585 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:27.342776 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-966823e3-e9db-4a51-a2c2-b471fbcc5585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:27.384981 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24abe204-c14c-40d5-8c9a-aed7bdb2f033 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:27.418600 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 52/96] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:27.419397 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966823e3-e9db-4a51-a2c2-b471fbcc5585 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:27.421985 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb26f30-97c3-4a2d-9fe1-c7a671541e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:27.422693 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb26f30-97c3-4a2d-9fe1-c7a671541e85 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:27.423042 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb26f30-97c3-4a2d-9fe1-c7a671541e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:27.423385 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb26f30-97c3-4a2d-9fe1-c7a671541e85 None admin] PATH_INFO: `/v3/users/6f317beb484644609d07d666527f760d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:27.424576 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb26f30-97c3-4a2d-9fe1-c7a671541e85 None admin] RBAC: Authorizing `identity:get_user(user_id=6f317beb484644609d07d666527f760d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:27.426783 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb26f30-97c3-4a2d-9fe1-c7a671541e85 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:27.428414 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 45/97] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:27 2022] GET /identity/v3/users/6f317beb484644609d07d666527f760d => generated 247 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:27.429793 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d746ea6-e990-4da2-a0f1-b9d76b2e5368 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:27.430139 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d746ea6-e990-4da2-a0f1-b9d76b2e5368 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:27.430478 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d746ea6-e990-4da2-a0f1-b9d76b2e5368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:27.450302 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-4d746ea6-e990-4da2-a0f1-b9d76b2e5368 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:27.451811 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-4d746ea6-e990-4da2-a0f1-b9d76b2e5368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:27.501032 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:27.527003 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 46/98] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:27.528544 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d8729-a190-4f34-b432-3186245a83c0 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:27.529065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d8729-a190-4f34-b432-3186245a83c0 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:27.529447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d8729-a190-4f34-b432-3186245a83c0 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:27.534791 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 47/99] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:27.534791 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d8729-a190-4f34-b432-3186245a83c0 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:27.542096 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966823e3-e9db-4a51-a2c2-b471fbcc5585 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:27.561683 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:27.580432 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a532d0a-f633-4ea3-bca4-12b8ecab2873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:27.581427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a532d0a-f633-4ea3-bca4-12b8ecab2873 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:27.582381 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a532d0a-f633-4ea3-bca4-12b8ecab2873 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:27.582890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a532d0a-f633-4ea3-bca4-12b8ecab2873 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:27.584028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a532d0a-f633-4ea3-bca4-12b8ecab2873 None admin] RBAC: Adding query filter params (name=member) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:27.584540 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a532d0a-f633-4ea3-bca4-12b8ecab2873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:27.586246 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a532d0a-f633-4ea3-bca4-12b8ecab2873 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:27.598460 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 53/100] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Oct 20 05:51:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:27.600930 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c0f3c-c251-4a06-8024-2dd818ab84b7 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:27.601406 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c0f3c-c251-4a06-8024-2dd818ab84b7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:27.601865 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c0f3c-c251-4a06-8024-2dd818ab84b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:27.630074 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-e47c0f3c-c251-4a06-8024-2dd818ab84b7 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:27.631577 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e47c0f3c-c251-4a06-8024-2dd818ab84b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:27.647637 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d8729-a190-4f34-b432-3186245a83c0 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:27.684549 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90abd619-4af7-479f-8ca4-d70190530bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:27.685455 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abd619-4af7-479f-8ca4-d70190530bfb None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:27.685880 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abd619-4af7-479f-8ca4-d70190530bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:27.686325 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abd619-4af7-479f-8ca4-d70190530bfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:27.741702 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 54/101] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:27.742859 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c0f3c-c251-4a06-8024-2dd818ab84b7 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:27.757921 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:27.811119 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90abd619-4af7-479f-8ca4-d70190530bfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:27.812608 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90abd619-4af7-479f-8ca4-d70190530bfb None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:27.928293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c0f3c-c251-4a06-8024-2dd818ab84b7 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:27.940440 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:27.945743 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 48/102] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:51:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 05:51:27.951029 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90abd619-4af7-479f-8ca4-d70190530bfb None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:27.969123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead3ccc-07f1-406b-b32c-99f5779ea38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:27.969857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead3ccc-07f1-406b-b32c-99f5779ea38f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:27.970140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead3ccc-07f1-406b-b32c-99f5779ea38f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:27.970413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead3ccc-07f1-406b-b32c-99f5779ea38f None admin] PATH_INFO: `/v3/users/3ca0dc0787674530a74f83cfa41df3d6` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:27.971547 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead3ccc-07f1-406b-b32c-99f5779ea38f None admin] RBAC: Authorizing `identity:get_user(user_id=3ca0dc0787674530a74f83cfa41df3d6)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:27.973550 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead3ccc-07f1-406b-b32c-99f5779ea38f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:27.975136 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 55/103] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:27 2022] GET /identity/v3/users/3ca0dc0787674530a74f83cfa41df3d6 => generated 275 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:27.976257 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e1651-0a5f-4d8c-83c9-e39600a70dc7 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:27.976539 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e1651-0a5f-4d8c-83c9-e39600a70dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:27.976839 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e1651-0a5f-4d8c-83c9-e39600a70dc7 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:27.978174 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 56/104] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:27.983106 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e1651-0a5f-4d8c-83c9-e39600a70dc7 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:28.043876 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:28.066936 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90abd619-4af7-479f-8ca4-d70190530bfb None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:28.087853 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e1651-0a5f-4d8c-83c9-e39600a70dc7 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:28.101255 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4defd9-e291-4508-a2c0-b6f0eabaefb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:28.101888 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4defd9-e291-4508-a2c0-b6f0eabaefb6 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:28.102165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4defd9-e291-4508-a2c0-b6f0eabaefb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:28.102430 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4defd9-e291-4508-a2c0-b6f0eabaefb6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:28.112489 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4defd9-e291-4508-a2c0-b6f0eabaefb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:28.117991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-3b4defd9-e291-4508-a2c0-b6f0eabaefb6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 05:51:28.117991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:28.117991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:28.117991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:28.117991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:28.117991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:28.117991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:28.117991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:28.117991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:28.117991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:28.117991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:28.117991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:28.117991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:28.117991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:28.117991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:28.117991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:28.119007 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:28.119007 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:28.119007 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:28.119007 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:28.119007 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:28.119007 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:28.119007 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:28.119007 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:28.119007 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:28.119007 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:28.119007 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:28.119007 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:28.119007 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:28.119007 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:28.119007 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:28.119007 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:28.121585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:28.121585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:28.121585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:28.121585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:28.121585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:28.121585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:28.121585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:28.121585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:28.121585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:28.121585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:28.121585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 05:51:28.121585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:51:28.121585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 49/105] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:51:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:51:28.121585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4defd9-e291-4508-a2c0-b6f0eabaefb6 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:28.125848 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7cf2f8-aad8-40b5-9eac-b97176dd1d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:28.126485 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7cf2f8-aad8-40b5-9eac-b97176dd1d6e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:28.126766 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7cf2f8-aad8-40b5-9eac-b97176dd1d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:28.127038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7cf2f8-aad8-40b5-9eac-b97176dd1d6e None admin] PATH_INFO: `/v3/projects/e78108de05fe409aa6e56461abb56cc0` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:28.128145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7cf2f8-aad8-40b5-9eac-b97176dd1d6e None admin] RBAC: Authorizing `identity:get_project(project_id=e78108de05fe409aa6e56461abb56cc0)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:28.129913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7cf2f8-aad8-40b5-9eac-b97176dd1d6e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:28.136494 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 57/106] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:51:27 2022] GET /identity/v3/projects/e78108de05fe409aa6e56461abb56cc0 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:28.136494 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d581676-84ee-4b52-984d-dab334136b4d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:28.136494 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d581676-84ee-4b52-984d-dab334136b4d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:28.136494 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d581676-84ee-4b52-984d-dab334136b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:28.151620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-9d581676-84ee-4b52-984d-dab334136b4d None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:28.152835 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9d581676-84ee-4b52-984d-dab334136b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:28.195718 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:28.220000 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 58/107] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:28.220804 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4defd9-e291-4508-a2c0-b6f0eabaefb6 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:28.221636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d581676-84ee-4b52-984d-dab334136b4d None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:28.258124 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e16ff8-858a-440f-8dea-52286aec85fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:28.259770 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e16ff8-858a-440f-8dea-52286aec85fa None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:28.260055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e16ff8-858a-440f-8dea-52286aec85fa None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:28.260332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e16ff8-858a-440f-8dea-52286aec85fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:28.268498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:28.273452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e16ff8-858a-440f-8dea-52286aec85fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:28.278452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-08e16ff8-858a-440f-8dea-52286aec85fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 05:51:28.278452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:28.278452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:28.278452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:28.278452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:28.278452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:28.278452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:28.278452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:28.278452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:28.278452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:28.278452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:28.278452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:28.278452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:28.278452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:28.278452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:28.278452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:28.279298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:28.279298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:28.279298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:28.279298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:28.279298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:28.279298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:28.279298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:28.279298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:28.279298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:28.279298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:28.279298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:28.279298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:28.279298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:28.279298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:28.279298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:28.279298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:28.280063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:28.280063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:28.280063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:28.280063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:28.280063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:28.280063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:28.280063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:28.280063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:28.280063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:28.280063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:28.280063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 05:51:28.280063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:51:28.280063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 50/108] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Oct 20 05:51:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:51:28.280685 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e16ff8-858a-440f-8dea-52286aec85fa None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:28.341157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d581676-84ee-4b52-984d-dab334136b4d None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:28.378353 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c66e7-a836-45ed-9d23-3c6ffe06cd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:28.379085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c66e7-a836-45ed-9d23-3c6ffe06cd77 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:28.379360 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c66e7-a836-45ed-9d23-3c6ffe06cd77 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:28.379642 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c66e7-a836-45ed-9d23-3c6ffe06cd77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:28.390897 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e16ff8-858a-440f-8dea-52286aec85fa None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:28.425266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:28.429356 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03bbb5-fde1-47c9-99a1-5bc11e10d5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:28.430034 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03bbb5-fde1-47c9-99a1-5bc11e10d5b4 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:28.430319 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03bbb5-fde1-47c9-99a1-5bc11e10d5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:28.430706 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03bbb5-fde1-47c9-99a1-5bc11e10d5b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:28.441328 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba03bbb5-fde1-47c9-99a1-5bc11e10d5b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:28.446719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-ba03bbb5-fde1-47c9-99a1-5bc11e10d5b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:51:28.446719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:28.446719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:28.446719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:28.446719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:28.446719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:28.446719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:28.446719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:28.446719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:28.446719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:28.446719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:28.446719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:28.446719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:28.446719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:28.446719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:28.446719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:28.447574 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:28.447574 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:28.447574 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:28.447574 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:28.447574 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:28.447574 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:28.447574 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:28.447574 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:28.447574 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:28.447574 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:28.447574 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:28.447574 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:28.447574 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:28.447574 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:28.447574 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:28.447574 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:28.448327 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:28.448327 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:28.448327 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:28.448327 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:28.448327 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:28.448327 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:28.448327 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:28.448327 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:28.448327 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:28.448327 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:28.448327 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:51:28.448327 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:51:28.448327 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 51/109] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:51:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:51:28.448990 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03bbb5-fde1-47c9-99a1-5bc11e10d5b4 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:28.455689 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c66e7-a836-45ed-9d23-3c6ffe06cd77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:28.458254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c66e7-a836-45ed-9d23-3c6ffe06cd77 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:28.514272 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:28.556206 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03bbb5-fde1-47c9-99a1-5bc11e10d5b4 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:28.557753 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 59/110] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:51:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 05:51:28.558619 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c66e7-a836-45ed-9d23-3c6ffe06cd77 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:28.592507 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ff3ed-9cc4-43cd-a98d-8b6af5b17e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:28.593144 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ff3ed-9cc4-43cd-a98d-8b6af5b17e87 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:28.593433 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ff3ed-9cc4-43cd-a98d-8b6af5b17e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:28.593711 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ff3ed-9cc4-43cd-a98d-8b6af5b17e87 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:28.594377 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ff3ed-9cc4-43cd-a98d-8b6af5b17e87 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:28.594689 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ff3ed-9cc4-43cd-a98d-8b6af5b17e87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:28.596093 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ff3ed-9cc4-43cd-a98d-8b6af5b17e87 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:28.604517 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 52/111] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Oct 20 05:51:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:28.609584 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ff3ed-9cc4-43cd-a98d-8b6af5b17e87 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:28.633039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:28.655855 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c66e7-a836-45ed-9d23-3c6ffe06cd77 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:28.687809 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9993147-4741-4570-80ba-347fb8de9385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:28.688445 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9993147-4741-4570-80ba-347fb8de9385 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:28.688748 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9993147-4741-4570-80ba-347fb8de9385 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:28.689022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9993147-4741-4570-80ba-347fb8de9385 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:28.689739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9993147-4741-4570-80ba-347fb8de9385 None admin] RBAC: Adding query filter params (role.id=d8debbb193834fac9cc25f81726f3a67, scope.project.id=e78108de05fe409aa6e56461abb56cc0, user.id=6f317beb484644609d07d666527f760d) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:28.690049 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9993147-4741-4570-80ba-347fb8de9385 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:28.692270 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9993147-4741-4570-80ba-347fb8de9385 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:28.699064 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 60/112] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:51:28 2022] GET /identity/v3/role_assignments?user.id=6f317beb484644609d07d666527f760d&scope.project.id=e78108de05fe409aa6e56461abb56cc0&role.id=d8debbb193834fac9cc25f81726f3a67 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:28.699770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9993147-4741-4570-80ba-347fb8de9385 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:28.710736 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ff3ed-9cc4-43cd-a98d-8b6af5b17e87 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:28.733481 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:28.750086 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e142db-7b8e-4b8b-bb9a-a34dae4237ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:28.750708 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e142db-7b8e-4b8b-bb9a-a34dae4237ce None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:28.750990 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e142db-7b8e-4b8b-bb9a-a34dae4237ce None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:28.751264 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e142db-7b8e-4b8b-bb9a-a34dae4237ce None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:28.751936 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e142db-7b8e-4b8b-bb9a-a34dae4237ce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:28.752236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e142db-7b8e-4b8b-bb9a-a34dae4237ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:28.753902 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e142db-7b8e-4b8b-bb9a-a34dae4237ce None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:28.764202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 53/113] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:51:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:28.765144 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e142db-7b8e-4b8b-bb9a-a34dae4237ce None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:28.829986 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9993147-4741-4570-80ba-347fb8de9385 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:28.870865 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3844f-5a1c-4783-ba2c-6660a3f4af2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:28.871478 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3844f-5a1c-4783-ba2c-6660a3f4af2d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:28.871756 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3844f-5a1c-4783-ba2c-6660a3f4af2d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:28.872026 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3844f-5a1c-4783-ba2c-6660a3f4af2d None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:28.872718 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3844f-5a1c-4783-ba2c-6660a3f4af2d None admin] RBAC: Adding query filter params (name=member) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:28.873018 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3844f-5a1c-4783-ba2c-6660a3f4af2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:28.874409 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3844f-5a1c-4783-ba2c-6660a3f4af2d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:28.880960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e142db-7b8e-4b8b-bb9a-a34dae4237ce None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:28.884143 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 61/114] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Oct 20 05:51:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:28.884761 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3844f-5a1c-4783-ba2c-6660a3f4af2d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:28.887361 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:28.918354 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a447569-53d2-4956-be91-df98fedbfc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:28.918998 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a447569-53d2-4956-be91-df98fedbfc2d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:28.919276 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a447569-53d2-4956-be91-df98fedbfc2d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:28.922108 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a447569-53d2-4956-be91-df98fedbfc2d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:28.922108 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a447569-53d2-4956-be91-df98fedbfc2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:28.922108 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a447569-53d2-4956-be91-df98fedbfc2d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:28.923836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 54/115] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:51:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:28.924445 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a447569-53d2-4956-be91-df98fedbfc2d None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:28.979830 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:28.997435 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3844f-5a1c-4783-ba2c-6660a3f4af2d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:29.029183 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a447569-53d2-4956-be91-df98fedbfc2d None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:29.030567 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28ea02-ef7d-4e31-ab9e-4d5f3030be7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:29.031090 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28ea02-ef7d-4e31-ab9e-4d5f3030be7b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:29.031374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28ea02-ef7d-4e31-ab9e-4d5f3030be7b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:29.031647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28ea02-ef7d-4e31-ab9e-4d5f3030be7b None admin] PATH_INFO: `/v3/projects/e78108de05fe409aa6e56461abb56cc0` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:29.032785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a28ea02-ef7d-4e31-ab9e-4d5f3030be7b None admin] RBAC: Authorizing `identity:get_project(project_id=e78108de05fe409aa6e56461abb56cc0)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:29.034220 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a28ea02-ef7d-4e31-ab9e-4d5f3030be7b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:29.036479 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 62/116] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:51:28 2022] GET /identity/v3/projects/e78108de05fe409aa6e56461abb56cc0 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:29.037637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28ea02-ef7d-4e31-ab9e-4d5f3030be7b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:29.063995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1a894-9e64-4a43-b9db-4b5aed5ed8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:29.064629 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1a894-9e64-4a43-b9db-4b5aed5ed8c7 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:29.064946 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1a894-9e64-4a43-b9db-4b5aed5ed8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:29.065220 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1a894-9e64-4a43-b9db-4b5aed5ed8c7 None admin] PATH_INFO: `/v3/users/0938b4beb5d34d0785a12d7c8d4edd64` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:29.080075 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1a894-9e64-4a43-b9db-4b5aed5ed8c7 None admin] RBAC: Authorizing `identity:get_user(user_id=0938b4beb5d34d0785a12d7c8d4edd64)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:29.081555 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1a894-9e64-4a43-b9db-4b5aed5ed8c7 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:29.084326 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 55/117] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:28 2022] GET /identity/v3/users/0938b4beb5d34d0785a12d7c8d4edd64 => generated 289 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:29.084968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1a894-9e64-4a43-b9db-4b5aed5ed8c7 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:29.107149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:29.138319 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28ea02-ef7d-4e31-ab9e-4d5f3030be7b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:29.171586 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30fbb1-a3df-449c-958f-bd0661886c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:29.172213 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30fbb1-a3df-449c-958f-bd0661886c7a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:29.172496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30fbb1-a3df-449c-958f-bd0661886c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:29.172804 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30fbb1-a3df-449c-958f-bd0661886c7a None admin] PATH_INFO: `/v3/users/3ca0dc0787674530a74f83cfa41df3d6` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:29.173932 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30fbb1-a3df-449c-958f-bd0661886c7a None admin] RBAC: Authorizing `identity:get_user(user_id=3ca0dc0787674530a74f83cfa41df3d6)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:29.175443 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30fbb1-a3df-449c-958f-bd0661886c7a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:29.177166 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 63/118] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:29 2022] GET /identity/v3/users/3ca0dc0787674530a74f83cfa41df3d6 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:29.177166 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30fbb1-a3df-449c-958f-bd0661886c7a None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:29.186824 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1a894-9e64-4a43-b9db-4b5aed5ed8c7 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:29.196202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:29.221892 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db9678-9627-4697-9481-a9705f6291d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:29.223438 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db9678-9627-4697-9481-a9705f6291d1 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:29.223798 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db9678-9627-4697-9481-a9705f6291d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:29.224074 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db9678-9627-4697-9481-a9705f6291d1 None admin] PATH_INFO: `/v3/users/3ca0dc0787674530a74f83cfa41df3d6` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:29.225253 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db9678-9627-4697-9481-a9705f6291d1 None admin] RBAC: Authorizing `identity:get_user(user_id=3ca0dc0787674530a74f83cfa41df3d6)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:29.226688 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db9678-9627-4697-9481-a9705f6291d1 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:29.228261 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 56/119] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:29 2022] GET /identity/v3/users/3ca0dc0787674530a74f83cfa41df3d6 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:29.228261 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db9678-9627-4697-9481-a9705f6291d1 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:29.276360 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30fbb1-a3df-449c-958f-bd0661886c7a None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:29.309148 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d94b5-85a4-40ba-a246-28cbf397bd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:29.309785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d94b5-85a4-40ba-a246-28cbf397bd62 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:29.310063 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d94b5-85a4-40ba-a246-28cbf397bd62 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:29.310334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d94b5-85a4-40ba-a246-28cbf397bd62 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:29.310974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d94b5-85a4-40ba-a246-28cbf397bd62 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:29.312433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d94b5-85a4-40ba-a246-28cbf397bd62 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:29.323039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:29.329754 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db9678-9627-4697-9481-a9705f6291d1 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:29.339889 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 64/120] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Oct 20 05:51:29 2022] POST /identity/v3/projects => generated 302 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:51:29.340913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d94b5-85a4-40ba-a246-28cbf397bd62 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:29.373902 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe832c-9855-46af-b465-8ebe8e69cce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:29.374625 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe832c-9855-46af-b465-8ebe8e69cce9 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:29.374973 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe832c-9855-46af-b465-8ebe8e69cce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:29.375323 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe832c-9855-46af-b465-8ebe8e69cce9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:29.376151 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe832c-9855-46af-b465-8ebe8e69cce9 None admin] RBAC: Adding query filter params (role.id=c1e0df53e1264ec79d91065e25acd29d, scope.project.id=e78108de05fe409aa6e56461abb56cc0, user.id=3ca0dc0787674530a74f83cfa41df3d6) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:29.376539 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe832c-9855-46af-b465-8ebe8e69cce9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:29.378134 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe832c-9855-46af-b465-8ebe8e69cce9 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:29.396735 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 57/121] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:51:29 2022] GET /identity/v3/role_assignments?user.id=3ca0dc0787674530a74f83cfa41df3d6&scope.project.id=e78108de05fe409aa6e56461abb56cc0&role.id=c1e0df53e1264ec79d91065e25acd29d => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:29.396735 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe832c-9855-46af-b465-8ebe8e69cce9 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:29.444399 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:29.471790 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d94b5-85a4-40ba-a246-28cbf397bd62 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:29.520562 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe832c-9855-46af-b465-8ebe8e69cce9 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:29.530997 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4896e1-1127-434d-83a5-3aeb8f496a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:29.532110 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4896e1-1127-434d-83a5-3aeb8f496a39 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:29.532398 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4896e1-1127-434d-83a5-3aeb8f496a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:29.532681 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4896e1-1127-434d-83a5-3aeb8f496a39 None admin] PATH_INFO: `/v3/projects/e78108de05fe409aa6e56461abb56cc0` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:29.534349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4896e1-1127-434d-83a5-3aeb8f496a39 None admin] RBAC: Authorizing `identity:get_project(project_id=e78108de05fe409aa6e56461abb56cc0)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:29.536680 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4896e1-1127-434d-83a5-3aeb8f496a39 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:29.539911 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 65/122] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:51:29 2022] GET /identity/v3/projects/e78108de05fe409aa6e56461abb56cc0 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:29.540335 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4896e1-1127-434d-83a5-3aeb8f496a39 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:29.569904 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acd3db-e890-4956-bad7-bc68f1e451d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:29.570556 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acd3db-e890-4956-bad7-bc68f1e451d2 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:29.570812 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acd3db-e890-4956-bad7-bc68f1e451d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:29.571083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acd3db-e890-4956-bad7-bc68f1e451d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:29.614734 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:29.658287 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15acd3db-e890-4956-bad7-bc68f1e451d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:29.660372 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15acd3db-e890-4956-bad7-bc68f1e451d2 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:29.667396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4896e1-1127-434d-83a5-3aeb8f496a39 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:29.709994 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3704f33-3c1a-422e-ad2a-d586b1a5752e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:29.710758 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3704f33-3c1a-422e-ad2a-d586b1a5752e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:29.711043 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3704f33-3c1a-422e-ad2a-d586b1a5752e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:29.711314 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3704f33-3c1a-422e-ad2a-d586b1a5752e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:29.724574 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:29.786452 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3704f33-3c1a-422e-ad2a-d586b1a5752e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:29.787861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3704f33-3c1a-422e-ad2a-d586b1a5752e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:29.788674 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 58/123] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:51:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 05:51:29.789407 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acd3db-e890-4956-bad7-bc68f1e451d2 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:29.855941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:29.897656 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acd3db-e890-4956-bad7-bc68f1e451d2 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:29.914337 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 66/124] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:51:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 05:51:29.915191 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3704f33-3c1a-422e-ad2a-d586b1a5752e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:29.936258 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec016ff-ac64-4793-ac2b-cea49b9e094a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:29.936955 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec016ff-ac64-4793-ac2b-cea49b9e094a None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:29.937234 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec016ff-ac64-4793-ac2b-cea49b9e094a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:29.937511 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec016ff-ac64-4793-ac2b-cea49b9e094a None admin] PATH_INFO: `/v3/projects/e78108de05fe409aa6e56461abb56cc0/users/0938b4beb5d34d0785a12d7c8d4edd64/roles/78820dc7bd3841cfbc80e594e4e77171` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:29.939389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec016ff-ac64-4793-ac2b-cea49b9e094a None admin] RBAC: Authorizing `identity:create_grant(project_id=e78108de05fe409aa6e56461abb56cc0, user_id=0938b4beb5d34d0785a12d7c8d4edd64, role_id=78820dc7bd3841cfbc80e594e4e77171)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:29.940876 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec016ff-ac64-4793-ac2b-cea49b9e094a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:29.951292 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:29.965555 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 59/125] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:51:29 2022] PUT /identity/v3/projects/e78108de05fe409aa6e56461abb56cc0/users/0938b4beb5d34d0785a12d7c8d4edd64/roles/78820dc7bd3841cfbc80e594e4e77171 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:51:29.966637 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec016ff-ac64-4793-ac2b-cea49b9e094a None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:30.033402 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3704f33-3c1a-422e-ad2a-d586b1a5752e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:30.075352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e91693-6d2f-4a09-a852-c6df1619364e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:30.075927 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e91693-6d2f-4a09-a852-c6df1619364e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:30.076213 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e91693-6d2f-4a09-a852-c6df1619364e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:30.076526 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e91693-6d2f-4a09-a852-c6df1619364e None admin] PATH_INFO: `/v3/projects/e78108de05fe409aa6e56461abb56cc0` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:30.077822 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e91693-6d2f-4a09-a852-c6df1619364e None admin] RBAC: Authorizing `identity:get_project(project_id=e78108de05fe409aa6e56461abb56cc0)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:30.079615 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e91693-6d2f-4a09-a852-c6df1619364e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:30.081260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 67/126] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:51:29 2022] GET /identity/v3/projects/e78108de05fe409aa6e56461abb56cc0 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:30.087955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e91693-6d2f-4a09-a852-c6df1619364e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:30.120100 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:30.125224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec016ff-ac64-4793-ac2b-cea49b9e094a None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:30.177125 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa460cea-aaff-4484-852c-b7d304fd0475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:30.177933 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa460cea-aaff-4484-852c-b7d304fd0475 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:30.178321 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa460cea-aaff-4484-852c-b7d304fd0475 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:30.178697 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa460cea-aaff-4484-852c-b7d304fd0475 None admin] PATH_INFO: `/v3/projects/883ca9f8306a4bcdaa646fb2f2b656a2` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:30.180459 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa460cea-aaff-4484-852c-b7d304fd0475 None admin] RBAC: Authorizing `identity:get_project(project_id=883ca9f8306a4bcdaa646fb2f2b656a2)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:30.182565 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa460cea-aaff-4484-852c-b7d304fd0475 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:30.184991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 60/127] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:51:29 2022] GET /identity/v3/projects/883ca9f8306a4bcdaa646fb2f2b656a2 => generated 312 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:30.191870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa460cea-aaff-4484-852c-b7d304fd0475 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:30.207112 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e91693-6d2f-4a09-a852-c6df1619364e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:30.241496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc9caa-3348-4c5e-abb2-3405b8efc141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:30.242337 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc9caa-3348-4c5e-abb2-3405b8efc141 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:30.242753 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc9caa-3348-4c5e-abb2-3405b8efc141 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:30.243163 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc9caa-3348-4c5e-abb2-3405b8efc141 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:30.244120 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc9caa-3348-4c5e-abb2-3405b8efc141 None admin] RBAC: Adding query filter params (role.id=49dae0405c764c96b7bfbcf9d9efd011, scope.project.id=e78108de05fe409aa6e56461abb56cc0, user.id=3ca0dc0787674530a74f83cfa41df3d6) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:30.244568 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc9caa-3348-4c5e-abb2-3405b8efc141 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:30.246153 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc9caa-3348-4c5e-abb2-3405b8efc141 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:30.252800 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 68/128] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:51:30 2022] GET /identity/v3/role_assignments?user.id=3ca0dc0787674530a74f83cfa41df3d6&scope.project.id=e78108de05fe409aa6e56461abb56cc0&role.id=49dae0405c764c96b7bfbcf9d9efd011 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:30.282427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:30.323691 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa460cea-aaff-4484-852c-b7d304fd0475 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:30.363792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f4be5-5f64-4d2b-bebc-260507ea0ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:30.364479 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f4be5-5f64-4d2b-bebc-260507ea0ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:30.364826 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f4be5-5f64-4d2b-bebc-260507ea0ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:30.365109 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f4be5-5f64-4d2b-bebc-260507ea0ba5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:30.366115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f4be5-5f64-4d2b-bebc-260507ea0ba5 None admin] RBAC: Adding query filter params (role.id=c1e0df53e1264ec79d91065e25acd29d, scope.project.id=883ca9f8306a4bcdaa646fb2f2b656a2, user.id=3ca0dc0787674530a74f83cfa41df3d6) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:30.366384 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f4be5-5f64-4d2b-bebc-260507ea0ba5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:30.367904 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f4be5-5f64-4d2b-bebc-260507ea0ba5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:30.375167 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 61/129] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:51:30 2022] GET /identity/v3/role_assignments?user.id=3ca0dc0787674530a74f83cfa41df3d6&scope.project.id=883ca9f8306a4bcdaa646fb2f2b656a2&role.id=c1e0df53e1264ec79d91065e25acd29d => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:31.592878 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f80d3f-7ab4-4fb2-b0b5-817c5d791c55 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:31.593366 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f80d3f-7ab4-4fb2-b0b5-817c5d791c55 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:31.593366 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f80d3f-7ab4-4fb2-b0b5-817c5d791c55 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:31.594839 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 69/130] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:31.600364 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90619b16-db92-45a7-91e2-ad4e33853901 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:31.600819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90619b16-db92-45a7-91e2-ad4e33853901 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:31.600955 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90619b16-db92-45a7-91e2-ad4e33853901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:31.619485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-90619b16-db92-45a7-91e2-ad4e33853901 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:31.620673 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-90619b16-db92-45a7-91e2-ad4e33853901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:31.685616 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 62/131] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 05:51:31.956079 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f80d3f-7ab4-4fb2-b0b5-817c5d791c55 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:32.048621 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:32.061138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f80d3f-7ab4-4fb2-b0b5-817c5d791c55 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:32.096991 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148a23b-ed66-46d8-9fd2-1d2bf97905d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:32.098154 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148a23b-ed66-46d8-9fd2-1d2bf97905d8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:32.098447 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148a23b-ed66-46d8-9fd2-1d2bf97905d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:32.098702 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148a23b-ed66-46d8-9fd2-1d2bf97905d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:32.099793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148a23b-ed66-46d8-9fd2-1d2bf97905d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:32.101276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148a23b-ed66-46d8-9fd2-1d2bf97905d8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:32.102991 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 70/132] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:51:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:32.108280 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90619b16-db92-45a7-91e2-ad4e33853901 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:32.163059 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf51c1a7-c513-4abf-b5d5-5b51e29ea25a None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:32.163356 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf51c1a7-c513-4abf-b5d5-5b51e29ea25a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:32.163687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf51c1a7-c513-4abf-b5d5-5b51e29ea25a None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:32.164990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 71/133] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:32.169903 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc498d6e-f4d6-4aca-b39c-02e61643a1e2 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:32.170185 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc498d6e-f4d6-4aca-b39c-02e61643a1e2 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:32.170455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc498d6e-f4d6-4aca-b39c-02e61643a1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:32.171907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:32.188195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-dc498d6e-f4d6-4aca-b39c-02e61643a1e2 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:32.189395 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-dc498d6e-f4d6-4aca-b39c-02e61643a1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:32.212151 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90619b16-db92-45a7-91e2-ad4e33853901 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:32.245342 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86725aa-4171-4476-95a4-cde32100edcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:32.245945 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86725aa-4171-4476-95a4-cde32100edcc None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:32.246234 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86725aa-4171-4476-95a4-cde32100edcc None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:32.246507 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86725aa-4171-4476-95a4-cde32100edcc None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:32.247153 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86725aa-4171-4476-95a4-cde32100edcc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:32.248608 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86725aa-4171-4476-95a4-cde32100edcc None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:32.254042 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 72/134] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:32.276451 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 63/135] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Oct 20 05:51:32 2022] POST /identity/v3/users => generated 283 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:51:32.530046 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc498d6e-f4d6-4aca-b39c-02e61643a1e2 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:32.622389 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:32.636968 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc498d6e-f4d6-4aca-b39c-02e61643a1e2 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:32.673203 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3bd7d-f877-4727-93ae-7da248040dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:32.673851 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3bd7d-f877-4727-93ae-7da248040dad None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:32.674133 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3bd7d-f877-4727-93ae-7da248040dad None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:32.674419 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3bd7d-f877-4727-93ae-7da248040dad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:32.685739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3bd7d-f877-4727-93ae-7da248040dad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:32.690676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-40a3bd7d-f877-4727-93ae-7da248040dad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 05:51:32.690676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:32.690676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:32.690676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:32.690676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:32.690676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:32.690676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:32.690676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:32.690676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:32.690676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:32.690676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:32.690676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:32.690676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:32.690676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:32.690676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:32.690676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:32.692469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:32.692469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:32.692469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:32.692469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:32.692469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:32.692469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:32.692469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:32.692469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:32.692469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:32.692469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:32.692469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:32.692469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:32.692469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:32.692469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:32.692469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:32.692469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:32.693427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:32.693427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:32.693427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:32.693427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:32.693427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:32.693427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:32.693427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:32.693427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:32.693427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:32.693427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:32.693427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 05:51:32.693427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:51:32.693427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 73/136] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:51:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:51:32.700486 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86725aa-4171-4476-95a4-cde32100edcc None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:32.810896 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86725aa-4171-4476-95a4-cde32100edcc None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:32.829535 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:32.849949 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae726e2-b655-4c9c-9e69-a31edcfd27a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:32.850570 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae726e2-b655-4c9c-9e69-a31edcfd27a5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:32.850848 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae726e2-b655-4c9c-9e69-a31edcfd27a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:32.851119 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae726e2-b655-4c9c-9e69-a31edcfd27a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:32.851786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae726e2-b655-4c9c-9e69-a31edcfd27a5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:32.852092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae726e2-b655-4c9c-9e69-a31edcfd27a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:32.853495 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae726e2-b655-4c9c-9e69-a31edcfd27a5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:32.862310 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 64/137] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Oct 20 05:51:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:32.868813 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3bd7d-f877-4727-93ae-7da248040dad None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:32.982230 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3bd7d-f877-4727-93ae-7da248040dad None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:33.019607 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b6c45-8f2b-4590-9c2f-faea4027d3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:33.020349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b6c45-8f2b-4590-9c2f-faea4027d3dd None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:33.020629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b6c45-8f2b-4590-9c2f-faea4027d3dd None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:33.020919 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b6c45-8f2b-4590-9c2f-faea4027d3dd None admin] PATH_INFO: `/v3/users/0938b4beb5d34d0785a12d7c8d4edd64` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:33.022051 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b6c45-8f2b-4590-9c2f-faea4027d3dd None admin] RBAC: Authorizing `identity:get_user(user_id=0938b4beb5d34d0785a12d7c8d4edd64)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:33.023559 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b6c45-8f2b-4590-9c2f-faea4027d3dd None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:33.030083 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 74/138] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:32 2022] GET /identity/v3/users/0938b4beb5d34d0785a12d7c8d4edd64 => generated 289 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:33.031659 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae726e2-b655-4c9c-9e69-a31edcfd27a5 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:33.139014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae726e2-b655-4c9c-9e69-a31edcfd27a5 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:33.174414 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60a19f-be84-44d0-a9a7-d66465b6bdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:33.175031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60a19f-be84-44d0-a9a7-d66465b6bdad None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:33.175341 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60a19f-be84-44d0-a9a7-d66465b6bdad None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:33.175671 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60a19f-be84-44d0-a9a7-d66465b6bdad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:33.227565 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:33.250815 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f60a19f-be84-44d0-a9a7-d66465b6bdad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:33.252765 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f60a19f-be84-44d0-a9a7-d66465b6bdad None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:33.360675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 65/139] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:51:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 05:51:33.369760 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b6c45-8f2b-4590-9c2f-faea4027d3dd None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:33.389795 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:33.479186 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b6c45-8f2b-4590-9c2f-faea4027d3dd None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:33.515214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad595a23-d180-4f67-95d4-c515301396c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:33.515847 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad595a23-d180-4f67-95d4-c515301396c6 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:33.516177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad595a23-d180-4f67-95d4-c515301396c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:33.516454 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad595a23-d180-4f67-95d4-c515301396c6 None admin] PATH_INFO: `/v3/projects/e78108de05fe409aa6e56461abb56cc0` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:33.517576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad595a23-d180-4f67-95d4-c515301396c6 None admin] RBAC: Authorizing `identity:get_project(project_id=e78108de05fe409aa6e56461abb56cc0)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:33.519010 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad595a23-d180-4f67-95d4-c515301396c6 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:33.520654 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 75/140] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:51:33 2022] GET /identity/v3/projects/e78108de05fe409aa6e56461abb56cc0 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:33.526794 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60a19f-be84-44d0-a9a7-d66465b6bdad None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:33.619241 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:33.640220 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60a19f-be84-44d0-a9a7-d66465b6bdad None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:33.676960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c3d5a-0181-4ba3-a0da-39e360072964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:33.677896 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c3d5a-0181-4ba3-a0da-39e360072964 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:33.678398 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c3d5a-0181-4ba3-a0da-39e360072964 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:33.678886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c3d5a-0181-4ba3-a0da-39e360072964 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:33.679815 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c3d5a-0181-4ba3-a0da-39e360072964 None admin] RBAC: Adding query filter params (role.id=78820dc7bd3841cfbc80e594e4e77171, scope.project.id=e78108de05fe409aa6e56461abb56cc0, user.id=0938b4beb5d34d0785a12d7c8d4edd64) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:33.680331 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c3d5a-0181-4ba3-a0da-39e360072964 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:33.682018 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c3d5a-0181-4ba3-a0da-39e360072964 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:33.688080 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 66/141] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:51:33 2022] GET /identity/v3/role_assignments?user.id=0938b4beb5d34d0785a12d7c8d4edd64&scope.project.id=e78108de05fe409aa6e56461abb56cc0&role.id=78820dc7bd3841cfbc80e594e4e77171 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:34.545214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d28c2e-33ac-4c76-9d50-4df758c2d185 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:34.545729 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d28c2e-33ac-4c76-9d50-4df758c2d185 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:34.545729 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d28c2e-33ac-4c76-9d50-4df758c2d185 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:34.547258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 76/142] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:34.552206 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3fc13-3255-4738-9ce8-a7a69c8e7e55 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:34.552610 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3fc13-3255-4738-9ce8-a7a69c8e7e55 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:34.552806 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3fc13-3255-4738-9ce8-a7a69c8e7e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:34.571349 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-81f3fc13-3255-4738-9ce8-a7a69c8e7e55 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:34.572468 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-81f3fc13-3255-4738-9ce8-a7a69c8e7e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:34.645593 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 67/143] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:34.719833 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ba64d-e2c0-4e51-838f-8bd9973c1a2f None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:34.720120 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ba64d-e2c0-4e51-838f-8bd9973c1a2f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:34.720555 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ba64d-e2c0-4e51-838f-8bd9973c1a2f None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:34.722691 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 77/144] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:34.728218 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ad140-6024-449a-a40a-a8baec41a921 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:34.728507 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ad140-6024-449a-a40a-a8baec41a921 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:34.728810 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ad140-6024-449a-a40a-a8baec41a921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:34.747954 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-eb1ad140-6024-449a-a40a-a8baec41a921 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:34.749174 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-eb1ad140-6024-449a-a40a-a8baec41a921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:34.821546 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 68/145] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:34.908017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbe1b9-2802-44ce-a420-61a498b8d3c9 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:34.908296 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbe1b9-2802-44ce-a420-61a498b8d3c9 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:34.908766 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbe1b9-2802-44ce-a420-61a498b8d3c9 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:34.910452 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 78/146] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:34.911213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ad140-6024-449a-a40a-a8baec41a921 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:34.914814 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:34.915188 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8a036-02c3-431c-980f-0f1b391909b6 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:34.915469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8a036-02c3-431c-980f-0f1b391909b6 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:34.915745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8a036-02c3-431c-980f-0f1b391909b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:34.934820 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a0d8a036-02c3-431c-980f-0f1b391909b6 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:34.936001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a0d8a036-02c3-431c-980f-0f1b391909b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:35.006145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 79/147] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 05:51:35.020688 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ad140-6024-449a-a40a-a8baec41a921 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:35.055012 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb8738-c43a-4b7c-9b43-942ded5fbe75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:35.055641 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb8738-c43a-4b7c-9b43-942ded5fbe75 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:35.055920 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb8738-c43a-4b7c-9b43-942ded5fbe75 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:35.056189 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb8738-c43a-4b7c-9b43-942ded5fbe75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:35.066251 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb8738-c43a-4b7c-9b43-942ded5fbe75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:35.071004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-cadb8738-c43a-4b7c-9b43-942ded5fbe75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:51:35.071004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:35.071004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:35.071004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:35.071004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:35.071004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:35.071004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:35.071004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:35.071004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:35.071004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:35.071004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:35.071004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:35.071004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:35.071004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:35.071004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:35.071004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:35.071907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:35.071907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:35.071907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:35.071907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:35.071907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:35.071907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:35.071907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:35.071907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:35.071907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:35.071907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:35.071907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:35.071907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:35.071907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:35.071907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:35.071907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:35.071907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:35.073631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:35.073631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:35.073631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:35.073631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:35.073631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:35.073631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:35.073631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:35.073631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:35.073631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:35.073631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:35.073631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:51:35.073631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:51:35.073631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 69/148] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Oct 20 05:51:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:51:35.077886 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8a036-02c3-431c-980f-0f1b391909b6 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:35.092897 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb8738-c43a-4b7c-9b43-942ded5fbe75 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:35.095304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:35.158972 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:35.185310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8a036-02c3-431c-980f-0f1b391909b6 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:35.208705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb8738-c43a-4b7c-9b43-942ded5fbe75 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:35.220619 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51eebf-e6b6-439d-92af-6ba5132305be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:35.221269 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51eebf-e6b6-439d-92af-6ba5132305be None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:35.221543 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51eebf-e6b6-439d-92af-6ba5132305be None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:35.221809 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51eebf-e6b6-439d-92af-6ba5132305be None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:35.223082 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51eebf-e6b6-439d-92af-6ba5132305be None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:35.223383 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51eebf-e6b6-439d-92af-6ba5132305be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:35.224802 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51eebf-e6b6-439d-92af-6ba5132305be None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:35.232675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 80/149] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:51:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:35.238137 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51eebf-e6b6-439d-92af-6ba5132305be None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:35.246362 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e211b-d201-41ed-a7cf-811409ee740d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:35.247224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e211b-d201-41ed-a7cf-811409ee740d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:35.247504 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e211b-d201-41ed-a7cf-811409ee740d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:35.247778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e211b-d201-41ed-a7cf-811409ee740d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:35.260015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e211b-d201-41ed-a7cf-811409ee740d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:35.265347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-3d7e211b-d201-41ed-a7cf-811409ee740d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 05:51:35.265347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:35.265347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:35.265347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:35.265347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:35.265347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:35.265347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:35.265347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:35.265347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:35.265347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:35.265347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:35.265347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:35.265347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:35.265347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:35.265347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:35.265347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:35.266197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:35.266197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:35.266197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:35.266197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:35.266197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:35.266197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:35.266197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:35.266197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:35.266197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:35.266197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:35.266197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:35.266197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:35.266197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:35.266197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:35.266197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:35.266197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:35.266970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:35.266970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:35.266970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:35.266970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:35.266970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:35.266970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:35.266970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:35.266970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:35.266970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:35.266970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:35.266970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 05:51:35.266970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:51:35.266970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 70/150] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Oct 20 05:51:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:51:35.272617 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e211b-d201-41ed-a7cf-811409ee740d None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:35.323705 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:35.346727 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51eebf-e6b6-439d-92af-6ba5132305be None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:35.374395 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e211b-d201-41ed-a7cf-811409ee740d None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:35.380085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10599aa4-0af8-43ad-a964-819e9de82e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:35.380738 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10599aa4-0af8-43ad-a964-819e9de82e95 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:35.381025 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10599aa4-0af8-43ad-a964-819e9de82e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:35.381311 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10599aa4-0af8-43ad-a964-819e9de82e95 None admin] PATH_INFO: `/v3/users/96f1c21e7675473f957bd2b29d128ead` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:35.394849 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10599aa4-0af8-43ad-a964-819e9de82e95 None admin] RBAC: Authorizing `identity:get_user(user_id=96f1c21e7675473f957bd2b29d128ead)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:35.396306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10599aa4-0af8-43ad-a964-819e9de82e95 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:35.397732 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:35.397956 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 81/151] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:35 2022] GET /identity/v3/users/96f1c21e7675473f957bd2b29d128ead => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:35.398713 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10599aa4-0af8-43ad-a964-819e9de82e95 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:35.409414 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f2dcf-8439-4c07-a02e-858159b28fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:35.411050 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f2dcf-8439-4c07-a02e-858159b28fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:35.411050 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f2dcf-8439-4c07-a02e-858159b28fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:35.411302 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f2dcf-8439-4c07-a02e-858159b28fe9 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:35.411959 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f2dcf-8439-4c07-a02e-858159b28fe9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:35.412392 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f2dcf-8439-4c07-a02e-858159b28fe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:35.413648 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f2dcf-8439-4c07-a02e-858159b28fe9 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:35.421534 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 71/152] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:51:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:35.423348 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f2dcf-8439-4c07-a02e-858159b28fe9 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:35.498233 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10599aa4-0af8-43ad-a964-819e9de82e95 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:35.525421 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f2dcf-8439-4c07-a02e-858159b28fe9 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:35.530854 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c115f-8861-45dc-9d35-1a376c831f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:35.531487 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c115f-8861-45dc-9d35-1a376c831f97 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:35.531767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c115f-8861-45dc-9d35-1a376c831f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:35.532045 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c115f-8861-45dc-9d35-1a376c831f97 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:35.533179 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c115f-8861-45dc-9d35-1a376c831f97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:35.535142 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c115f-8861-45dc-9d35-1a376c831f97 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:35.537111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 82/153] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:51:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:35.537684 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c115f-8861-45dc-9d35-1a376c831f97 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:35.540265 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:35.562359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076aba7-0776-4000-848d-2cb0a9cd0b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:35.562991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076aba7-0776-4000-848d-2cb0a9cd0b6a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:35.563263 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076aba7-0776-4000-848d-2cb0a9cd0b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:35.563529 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076aba7-0776-4000-848d-2cb0a9cd0b6a None admin] PATH_INFO: `/v3/projects/e5ab74dfb5014b01bea8909bd1d64559` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:35.564622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9076aba7-0776-4000-848d-2cb0a9cd0b6a None admin] RBAC: Authorizing `identity:get_project(project_id=e5ab74dfb5014b01bea8909bd1d64559)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:35.566109 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9076aba7-0776-4000-848d-2cb0a9cd0b6a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:35.567862 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 72/154] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:51:35 2022] GET /identity/v3/projects/e5ab74dfb5014b01bea8909bd1d64559 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:35.568437 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076aba7-0776-4000-848d-2cb0a9cd0b6a None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:35.625107 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:35.637144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c115f-8861-45dc-9d35-1a376c831f97 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:35.669205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076aba7-0776-4000-848d-2cb0a9cd0b6a None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:35.669384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2e39f-940c-47e1-849e-7bdd096aff44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:35.669831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2e39f-940c-47e1-849e-7bdd096aff44 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:35.670111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2e39f-940c-47e1-849e-7bdd096aff44 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:35.670388 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2e39f-940c-47e1-849e-7bdd096aff44 None admin] PATH_INFO: `/v3/users/96f1c21e7675473f957bd2b29d128ead` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:35.671516 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2e39f-940c-47e1-849e-7bdd096aff44 None admin] RBAC: Authorizing `identity:get_user(user_id=96f1c21e7675473f957bd2b29d128ead)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:35.672998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2e39f-940c-47e1-849e-7bdd096aff44 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:35.674576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 83/155] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:35 2022] GET /identity/v3/users/96f1c21e7675473f957bd2b29d128ead => generated 283 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:35.679685 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2e39f-940c-47e1-849e-7bdd096aff44 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:35.706235 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ed747-346e-42ce-bcc0-2f308820e2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:35.707346 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ed747-346e-42ce-bcc0-2f308820e2bb None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:35.707623 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ed747-346e-42ce-bcc0-2f308820e2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:35.707892 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ed747-346e-42ce-bcc0-2f308820e2bb None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:35.708531 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ed747-346e-42ce-bcc0-2f308820e2bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:35.709998 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ed747-346e-42ce-bcc0-2f308820e2bb None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:35.730845 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 73/156] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Oct 20 05:51:35 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:51:35.731658 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ed747-346e-42ce-bcc0-2f308820e2bb None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:35.753839 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:35.799956 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2e39f-940c-47e1-849e-7bdd096aff44 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:35.844445 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5be23-9e64-4757-b12b-0566f9d7861b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:35.845156 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5be23-9e64-4757-b12b-0566f9d7861b None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:35.845437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5be23-9e64-4757-b12b-0566f9d7861b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:35.845712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5be23-9e64-4757-b12b-0566f9d7861b None admin] PATH_INFO: `/v3/projects/e5ab74dfb5014b01bea8909bd1d64559/users/96f1c21e7675473f957bd2b29d128ead/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:35.848832 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5be23-9e64-4757-b12b-0566f9d7861b None admin] RBAC: Authorizing `identity:create_grant(project_id=e5ab74dfb5014b01bea8909bd1d64559, user_id=96f1c21e7675473f957bd2b29d128ead, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:35.850363 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5be23-9e64-4757-b12b-0566f9d7861b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:35.850634 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ed747-346e-42ce-bcc0-2f308820e2bb None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:35.864067 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:35.867010 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 84/157] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:51:35 2022] PUT /identity/v3/projects/e5ab74dfb5014b01bea8909bd1d64559/users/96f1c21e7675473f957bd2b29d128ead/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:51:35.893671 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b8d1e-c228-4d9c-8927-9c4e7602b235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:35.894865 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b8d1e-c228-4d9c-8927-9c4e7602b235 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:35.895140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b8d1e-c228-4d9c-8927-9c4e7602b235 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:35.895411 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b8d1e-c228-4d9c-8927-9c4e7602b235 None admin] PATH_INFO: `/v3/projects/e5ab74dfb5014b01bea8909bd1d64559` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:35.896754 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b8d1e-c228-4d9c-8927-9c4e7602b235 None admin] RBAC: Authorizing `identity:get_project(project_id=e5ab74dfb5014b01bea8909bd1d64559)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:35.898251 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b8d1e-c228-4d9c-8927-9c4e7602b235 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:35.900169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 74/158] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:51:35 2022] GET /identity/v3/projects/e5ab74dfb5014b01bea8909bd1d64559 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:35.905927 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5be23-9e64-4757-b12b-0566f9d7861b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:36.033254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5be23-9e64-4757-b12b-0566f9d7861b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:36.065573 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:36.074311 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e81328-5b2f-4ac0-961d-db2af75ed0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:36.074976 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e81328-5b2f-4ac0-961d-db2af75ed0b6 None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:36.075254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e81328-5b2f-4ac0-961d-db2af75ed0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:36.075526 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e81328-5b2f-4ac0-961d-db2af75ed0b6 None admin] PATH_INFO: `/v3/projects/e5ab74dfb5014b01bea8909bd1d64559/users/96f1c21e7675473f957bd2b29d128ead/roles/49dae0405c764c96b7bfbcf9d9efd011` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:36.077728 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e81328-5b2f-4ac0-961d-db2af75ed0b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5ab74dfb5014b01bea8909bd1d64559, user_id=96f1c21e7675473f957bd2b29d128ead, role_id=49dae0405c764c96b7bfbcf9d9efd011)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:36.079177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e81328-5b2f-4ac0-961d-db2af75ed0b6 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:36.093300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 85/159] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:51:35 2022] PUT /identity/v3/projects/e5ab74dfb5014b01bea8909bd1d64559/users/96f1c21e7675473f957bd2b29d128ead/roles/49dae0405c764c96b7bfbcf9d9efd011 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:51:38.126041 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77371fd-921b-4e27-a059-9833df63ffe6 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:38.126595 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77371fd-921b-4e27-a059-9833df63ffe6 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:38.126682 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77371fd-921b-4e27-a059-9833df63ffe6 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:38.128037 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 75/160] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:38.134375 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948b71a-1029-4594-adf0-9269d9b38f6d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:38.134906 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948b71a-1029-4594-adf0-9269d9b38f6d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:38.134906 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948b71a-1029-4594-adf0-9269d9b38f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:38.155205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-3948b71a-1029-4594-adf0-9269d9b38f6d None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:38.156421 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3948b71a-1029-4594-adf0-9269d9b38f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:38.194240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4257a-5495-40bd-9300-acc517496744 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:38.194561 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4257a-5495-40bd-9300-acc517496744 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:38.194792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4257a-5495-40bd-9300-acc517496744 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:38.196132 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 76/161] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:51:38.201453 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2d79a-0919-4f76-bf47-45595c0e73b0 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:38.201848 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2d79a-0919-4f76-bf47-45595c0e73b0 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:38.202228 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2d79a-0919-4f76-bf47-45595c0e73b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:38.221369 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-a3f2d79a-0919-4f76-bf47-45595c0e73b0 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:38.223586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a3f2d79a-0919-4f76-bf47-45595c0e73b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:38.237267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 86/162] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:38.244635 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd6d2c-6154-477c-a59d-bd91d6ac8d35 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:38.244951 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd6d2c-6154-477c-a59d-bd91d6ac8d35 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:38.245227 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd6d2c-6154-477c-a59d-bd91d6ac8d35 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:38.246538 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 87/163] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:51:38.251855 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e06cbe-4210-4bf0-afd1-4a82e8c6ff00 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:38.252136 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e06cbe-4210-4bf0-afd1-4a82e8c6ff00 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:38.252411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e06cbe-4210-4bf0-afd1-4a82e8c6ff00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:38.273587 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-f0e06cbe-4210-4bf0-afd1-4a82e8c6ff00 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:38.274819 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f0e06cbe-4210-4bf0-afd1-4a82e8c6ff00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:38.310812 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 77/164] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 05:51:38.359578 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 88/165] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 05:51:38.394977 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05743e-2f22-45a7-9850-414a5f231eca None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:38.395257 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05743e-2f22-45a7-9850-414a5f231eca None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:38.395541 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05743e-2f22-45a7-9850-414a5f231eca None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:38.397178 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 78/166] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:38.402259 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729b640-9abf-41df-be94-091617fe8b02 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:38.402543 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729b640-9abf-41df-be94-091617fe8b02 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:38.402817 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729b640-9abf-41df-be94-091617fe8b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:38.421533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-6729b640-9abf-41df-be94-091617fe8b02 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:38.422852 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6729b640-9abf-41df-be94-091617fe8b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:38.470494 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:38.493553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 89/167] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:38.529811 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05743e-2f22-45a7-9850-414a5f231eca None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:38.586761 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729b640-9abf-41df-be94-091617fe8b02 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:38.635209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05743e-2f22-45a7-9850-414a5f231eca None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:38.645196 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:38.670867 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d02c5-02fb-4a14-a035-ec0d33fe2a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:38.671500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d02c5-02fb-4a14-a035-ec0d33fe2a99 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:38.671868 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d02c5-02fb-4a14-a035-ec0d33fe2a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:38.672049 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d02c5-02fb-4a14-a035-ec0d33fe2a99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:38.682735 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d02c5-02fb-4a14-a035-ec0d33fe2a99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:38.687533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-4d0d02c5-02fb-4a14-a035-ec0d33fe2a99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:51:38.687533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:38.687533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:38.687533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:38.687533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:38.687533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:38.687533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:38.687533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:38.687533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:38.687533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:38.687533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:38.687533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:38.687533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:38.687533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:38.687533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:38.687533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:38.688492 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:38.688492 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:38.688492 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:38.688492 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:38.688492 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:38.688492 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:38.688492 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:38.688492 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:38.688492 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:38.688492 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:38.688492 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:38.688492 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:38.688492 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:38.688492 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:38.688492 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:38.688492 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:38.689373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:38.689373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:38.689373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:38.689373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:38.689373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:38.689373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:38.689373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:38.689373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:38.689373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:38.689373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:38.689373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:51:38.689373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:51:38.689373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 79/168] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Oct 20 05:51:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:51:38.689373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d02c5-02fb-4a14-a035-ec0d33fe2a99 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:38.710407 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729b640-9abf-41df-be94-091617fe8b02 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:38.745570 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b67ef-1995-44f0-bba5-b6718c00d620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:38.746361 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b67ef-1995-44f0-bba5-b6718c00d620 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:38.746642 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b67ef-1995-44f0-bba5-b6718c00d620 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:38.746917 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b67ef-1995-44f0-bba5-b6718c00d620 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:38.758037 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b67ef-1995-44f0-bba5-b6718c00d620 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:38.763657 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-156b67ef-1995-44f0-bba5-b6718c00d620 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:51:38.763657 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:38.763657 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:38.763657 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:38.763657 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:38.763657 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:38.763657 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:38.763657 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:38.763657 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:38.763657 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:38.763657 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:38.763657 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:38.763657 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:38.763657 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:38.763657 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:38.763657 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:38.764533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:38.764533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:38.764533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:38.764533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:38.764533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:38.764533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:38.764533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:38.764533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:38.764533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:38.764533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:38.764533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:38.764533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:38.764533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:38.764533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:38.764533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:38.764533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:38.765629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:38.765629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:38.765629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:38.765629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:38.765629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:38.765629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:38.765629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:38.765629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:38.765629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:38.765629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:38.765629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:51:38.765629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:51:38.765629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 90/169] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:51:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:51:38.765629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b67ef-1995-44f0-bba5-b6718c00d620 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:38.797631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d02c5-02fb-4a14-a035-ec0d33fe2a99 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:38.809850 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:38.832225 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f476631-437e-4ed9-98f0-44579e5c9b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:38.832884 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f476631-437e-4ed9-98f0-44579e5c9b72 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:38.833164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f476631-437e-4ed9-98f0-44579e5c9b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:38.833431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f476631-437e-4ed9-98f0-44579e5c9b72 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:38.834553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f476631-437e-4ed9-98f0-44579e5c9b72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:38.836011 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f476631-437e-4ed9-98f0-44579e5c9b72 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:38.837604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 80/170] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:51:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:38.838367 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f476631-437e-4ed9-98f0-44579e5c9b72 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:38.872300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b67ef-1995-44f0-bba5-b6718c00d620 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:38.873598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:38.906350 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e73486-4e31-4aab-bb9a-0e2f0f17463d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:38.906883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e73486-4e31-4aab-bb9a-0e2f0f17463d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:38.907159 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e73486-4e31-4aab-bb9a-0e2f0f17463d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:38.907426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e73486-4e31-4aab-bb9a-0e2f0f17463d None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:38.908080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e73486-4e31-4aab-bb9a-0e2f0f17463d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:38.908380 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e73486-4e31-4aab-bb9a-0e2f0f17463d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:38.910564 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e73486-4e31-4aab-bb9a-0e2f0f17463d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:38.918887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 91/171] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:51:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:38.919505 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e73486-4e31-4aab-bb9a-0e2f0f17463d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:38.946984 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f476631-437e-4ed9-98f0-44579e5c9b72 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:38.981094 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647e97c-9977-4ffc-9c3e-6e336fde725c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:38.981698 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647e97c-9977-4ffc-9c3e-6e336fde725c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:38.981974 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647e97c-9977-4ffc-9c3e-6e336fde725c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:38.982246 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647e97c-9977-4ffc-9c3e-6e336fde725c None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:38.982895 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647e97c-9977-4ffc-9c3e-6e336fde725c None admin] RBAC: Adding query filter params (name=member) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:38.983196 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647e97c-9977-4ffc-9c3e-6e336fde725c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:38.984570 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647e97c-9977-4ffc-9c3e-6e336fde725c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:38.993724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 81/172] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Oct 20 05:51:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:38.994498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647e97c-9977-4ffc-9c3e-6e336fde725c None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:39.023743 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e73486-4e31-4aab-bb9a-0e2f0f17463d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:39.025245 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:39.058445 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2ec51-24eb-4a35-b98d-2c2e8d3f2fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:39.059073 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2ec51-24eb-4a35-b98d-2c2e8d3f2fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:39.059348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2ec51-24eb-4a35-b98d-2c2e8d3f2fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:39.059618 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2ec51-24eb-4a35-b98d-2c2e8d3f2fc9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:39.071172 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2ec51-24eb-4a35-b98d-2c2e8d3f2fc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:39.076044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-f6b2ec51-24eb-4a35-b98d-2c2e8d3f2fc9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 05:51:39.076044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:39.076044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:39.076044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:39.076044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:39.076044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:39.076044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:39.076044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:39.076044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:39.076044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:39.076044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:39.076044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:39.076044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:39.076044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:39.076044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:39.076044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:39.076992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:39.076992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:39.076992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:39.076992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:39.076992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:39.076992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:39.076992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:39.076992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:39.076992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:39.076992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:39.076992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:39.076992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:39.076992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:39.076992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:39.076992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:39.076992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:39.077764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:39.077764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:39.077764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:39.077764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:39.077764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:39.077764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:39.077764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:39.077764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:39.077764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:39.077764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:39.077764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 05:51:39.077764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:51:39.077764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 92/173] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Oct 20 05:51:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:51:39.078422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2ec51-24eb-4a35-b98d-2c2e8d3f2fc9 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:39.092007 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:39.096361 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647e97c-9977-4ffc-9c3e-6e336fde725c None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:39.131103 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1175933-97e3-4a39-8d38-3f1b230e905a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:39.131779 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1175933-97e3-4a39-8d38-3f1b230e905a None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:39.131980 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1175933-97e3-4a39-8d38-3f1b230e905a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:39.132248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1175933-97e3-4a39-8d38-3f1b230e905a None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:39.132927 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1175933-97e3-4a39-8d38-3f1b230e905a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:39.134629 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1175933-97e3-4a39-8d38-3f1b230e905a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:39.155753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 82/174] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Oct 20 05:51:38 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:51:39.156626 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1175933-97e3-4a39-8d38-3f1b230e905a None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:39.194345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2ec51-24eb-4a35-b98d-2c2e8d3f2fc9 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:39.233363 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0916bcf-8af0-4a8d-a9db-aa07c9565950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:39.234406 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0916bcf-8af0-4a8d-a9db-aa07c9565950 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:39.234843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0916bcf-8af0-4a8d-a9db-aa07c9565950 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:39.235338 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0916bcf-8af0-4a8d-a9db-aa07c9565950 None admin] PATH_INFO: `/v3/users/96f1c21e7675473f957bd2b29d128ead` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:39.236750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0916bcf-8af0-4a8d-a9db-aa07c9565950 None admin] RBAC: Authorizing `identity:get_user(user_id=96f1c21e7675473f957bd2b29d128ead)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:39.239018 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0916bcf-8af0-4a8d-a9db-aa07c9565950 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:39.241134 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 93/175] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:39 2022] GET /identity/v3/users/96f1c21e7675473f957bd2b29d128ead => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:39.241765 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0916bcf-8af0-4a8d-a9db-aa07c9565950 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:39.278222 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:39.291218 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1175933-97e3-4a39-8d38-3f1b230e905a None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:39.332446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d2928-4bf0-45df-886e-632e7e5eb61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:39.333419 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d2928-4bf0-45df-886e-632e7e5eb61d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:39.333862 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d2928-4bf0-45df-886e-632e7e5eb61d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:39.334262 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d2928-4bf0-45df-886e-632e7e5eb61d None admin] PATH_INFO: `/v3/users/f944919f7f08497dbad4b41cf82a240b` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:39.351558 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d2928-4bf0-45df-886e-632e7e5eb61d None admin] RBAC: Authorizing `identity:get_user(user_id=f944919f7f08497dbad4b41cf82a240b)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:39.353495 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d2928-4bf0-45df-886e-632e7e5eb61d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:39.354733 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 83/176] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:39 2022] GET /identity/v3/users/f944919f7f08497dbad4b41cf82a240b => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:39.357682 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d2928-4bf0-45df-886e-632e7e5eb61d None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:39.363588 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0916bcf-8af0-4a8d-a9db-aa07c9565950 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:39.377278 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:39.407729 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47cd94-37fe-443c-ada3-8ad0c53a3917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:39.408574 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47cd94-37fe-443c-ada3-8ad0c53a3917 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:39.408900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47cd94-37fe-443c-ada3-8ad0c53a3917 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:39.409171 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47cd94-37fe-443c-ada3-8ad0c53a3917 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:39.410450 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a47cd94-37fe-443c-ada3-8ad0c53a3917 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:39.410750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a47cd94-37fe-443c-ada3-8ad0c53a3917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:39.412120 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a47cd94-37fe-443c-ada3-8ad0c53a3917 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:39.420572 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 94/177] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:51:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:39.421232 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47cd94-37fe-443c-ada3-8ad0c53a3917 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:39.481909 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d2928-4bf0-45df-886e-632e7e5eb61d None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:39.531593 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b23b5-b956-46a0-bb46-840d67a37a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:39.532215 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b23b5-b956-46a0-bb46-840d67a37a68 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:39.532494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b23b5-b956-46a0-bb46-840d67a37a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:39.532792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b23b5-b956-46a0-bb46-840d67a37a68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:39.541885 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:39.557837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47cd94-37fe-443c-ada3-8ad0c53a3917 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:39.605908 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b302a5-35fa-4077-aa2f-5c55969e04d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:39.606919 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b302a5-35fa-4077-aa2f-5c55969e04d4 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:39.607409 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b302a5-35fa-4077-aa2f-5c55969e04d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:39.608094 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b302a5-35fa-4077-aa2f-5c55969e04d4 None admin] PATH_INFO: `/v3/projects/e5ab74dfb5014b01bea8909bd1d64559` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:39.610036 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b302a5-35fa-4077-aa2f-5c55969e04d4 None admin] RBAC: Authorizing `identity:get_project(project_id=e5ab74dfb5014b01bea8909bd1d64559)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:39.611272 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b23b5-b956-46a0-bb46-840d67a37a68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:39.612198 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b302a5-35fa-4077-aa2f-5c55969e04d4 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:39.613520 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b23b5-b956-46a0-bb46-840d67a37a68 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:39.614823 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 95/178] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:51:39 2022] GET /identity/v3/projects/e5ab74dfb5014b01bea8909bd1d64559 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:39.616095 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b302a5-35fa-4077-aa2f-5c55969e04d4 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:39.629939 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:39.724774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 84/179] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:51:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 05:51:39.725555 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b23b5-b956-46a0-bb46-840d67a37a68 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:39.762559 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b302a5-35fa-4077-aa2f-5c55969e04d4 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:39.804824 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a1ee8-4330-45c4-96b7-64ca59005ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:39.805174 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a1ee8-4330-45c4-96b7-64ca59005ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:39.805425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a1ee8-4330-45c4-96b7-64ca59005ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:39.805697 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a1ee8-4330-45c4-96b7-64ca59005ee7 None admin] PATH_INFO: `/v3/users/96f1c21e7675473f957bd2b29d128ead` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:39.807507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a1ee8-4330-45c4-96b7-64ca59005ee7 None admin] RBAC: Authorizing `identity:get_user(user_id=96f1c21e7675473f957bd2b29d128ead)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:39.808999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a1ee8-4330-45c4-96b7-64ca59005ee7 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:39.811350 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 96/180] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:39 2022] GET /identity/v3/users/96f1c21e7675473f957bd2b29d128ead => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:39.811945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a1ee8-4330-45c4-96b7-64ca59005ee7 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:39.844855 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:39.855979 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b23b5-b956-46a0-bb46-840d67a37a68 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:39.881048 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:39.905413 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62373f83-5479-4444-bf54-24f50a4e05f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:39.906594 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62373f83-5479-4444-bf54-24f50a4e05f6 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:39.907314 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62373f83-5479-4444-bf54-24f50a4e05f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:39.907755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62373f83-5479-4444-bf54-24f50a4e05f6 None admin] PATH_INFO: `/v3/projects/e5ab74dfb5014b01bea8909bd1d64559/users/f944919f7f08497dbad4b41cf82a240b/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:39.910778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62373f83-5479-4444-bf54-24f50a4e05f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5ab74dfb5014b01bea8909bd1d64559, user_id=f944919f7f08497dbad4b41cf82a240b, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:39.912969 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62373f83-5479-4444-bf54-24f50a4e05f6 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:39.925842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a1ee8-4330-45c4-96b7-64ca59005ee7 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:39.931001 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 85/181] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:51:39 2022] PUT /identity/v3/projects/e5ab74dfb5014b01bea8909bd1d64559/users/f944919f7f08497dbad4b41cf82a240b/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:51:39.932117 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62373f83-5479-4444-bf54-24f50a4e05f6 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:39.974491 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd2d76-834d-465e-a904-f9a5adbd9379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:39.976320 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd2d76-834d-465e-a904-f9a5adbd9379 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:39.976994 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd2d76-834d-465e-a904-f9a5adbd9379 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:39.977438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd2d76-834d-465e-a904-f9a5adbd9379 None admin] PATH_INFO: `/v3/projects/e5ab74dfb5014b01bea8909bd1d64559` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:39.979013 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd2d76-834d-465e-a904-f9a5adbd9379 None admin] RBAC: Authorizing `identity:get_project(project_id=e5ab74dfb5014b01bea8909bd1d64559)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:39.981672 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd2d76-834d-465e-a904-f9a5adbd9379 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:39.983650 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 97/182] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:51:39 2022] GET /identity/v3/projects/e5ab74dfb5014b01bea8909bd1d64559 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:39.989656 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd2d76-834d-465e-a904-f9a5adbd9379 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:40.078149 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62373f83-5479-4444-bf54-24f50a4e05f6 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:40.141671 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a8514-295d-49ff-8b44-f42b0389af1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:40.142293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a8514-295d-49ff-8b44-f42b0389af1b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:40.142557 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a8514-295d-49ff-8b44-f42b0389af1b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:40.142845 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a8514-295d-49ff-8b44-f42b0389af1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:40.176384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:40.187011 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd2d76-834d-465e-a904-f9a5adbd9379 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:40.205417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:40.236290 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9590fa52-8b48-490e-add8-2a951387ecf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:40.236828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590fa52-8b48-490e-add8-2a951387ecf3 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:40.237130 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590fa52-8b48-490e-add8-2a951387ecf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:40.237350 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590fa52-8b48-490e-add8-2a951387ecf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:40.238640 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9590fa52-8b48-490e-add8-2a951387ecf3 None admin] RBAC: Adding query filter params (role.id=d8debbb193834fac9cc25f81726f3a67, scope.project.id=e5ab74dfb5014b01bea8909bd1d64559, user.id=96f1c21e7675473f957bd2b29d128ead) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:40.239060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9590fa52-8b48-490e-add8-2a951387ecf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:40.240507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9590fa52-8b48-490e-add8-2a951387ecf3 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:40.245373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a8514-295d-49ff-8b44-f42b0389af1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:40.247083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a8514-295d-49ff-8b44-f42b0389af1b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:40.260729 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 98/183] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:51:39 2022] GET /identity/v3/role_assignments?user.id=96f1c21e7675473f957bd2b29d128ead&scope.project.id=e5ab74dfb5014b01bea8909bd1d64559&role.id=d8debbb193834fac9cc25f81726f3a67 => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:40.377479 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 86/184] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:51:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 05:51:40.383839 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9590fa52-8b48-490e-add8-2a951387ecf3 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:40.520344 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9590fa52-8b48-490e-add8-2a951387ecf3 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:40.562791 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac551eea-9577-42f6-9574-a652440b9bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:40.563425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac551eea-9577-42f6-9574-a652440b9bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:40.563701 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac551eea-9577-42f6-9574-a652440b9bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:40.563970 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac551eea-9577-42f6-9574-a652440b9bd0 None admin] PATH_INFO: `/v3/projects/e5ab74dfb5014b01bea8909bd1d64559` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:40.565111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac551eea-9577-42f6-9574-a652440b9bd0 None admin] RBAC: Authorizing `identity:get_project(project_id=e5ab74dfb5014b01bea8909bd1d64559)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:40.567053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac551eea-9577-42f6-9574-a652440b9bd0 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:40.569012 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 99/185] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:51:40 2022] GET /identity/v3/projects/e5ab74dfb5014b01bea8909bd1d64559 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:40.574848 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a8514-295d-49ff-8b44-f42b0389af1b None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:40.656920 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:40.688859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a8514-295d-49ff-8b44-f42b0389af1b None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:40.734776 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb783b4-2c78-4b6b-8a1a-4276c2d65651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:40.735405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb783b4-2c78-4b6b-8a1a-4276c2d65651 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:40.735685 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb783b4-2c78-4b6b-8a1a-4276c2d65651 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:40.735954 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb783b4-2c78-4b6b-8a1a-4276c2d65651 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:40.736684 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb783b4-2c78-4b6b-8a1a-4276c2d65651 None admin] RBAC: Adding query filter params (role.id=49dae0405c764c96b7bfbcf9d9efd011, scope.project.id=e5ab74dfb5014b01bea8909bd1d64559, user.id=96f1c21e7675473f957bd2b29d128ead) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:40.737016 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb783b4-2c78-4b6b-8a1a-4276c2d65651 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:40.738584 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb783b4-2c78-4b6b-8a1a-4276c2d65651 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:40.744784 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 87/186] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:51:40 2022] GET /identity/v3/role_assignments?user.id=96f1c21e7675473f957bd2b29d128ead&scope.project.id=e5ab74dfb5014b01bea8909bd1d64559&role.id=49dae0405c764c96b7bfbcf9d9efd011 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:41.585625 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31ac7d-60d6-4978-aa94-cd44801747fd None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:41.586129 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31ac7d-60d6-4978-aa94-cd44801747fd None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:41.586129 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31ac7d-60d6-4978-aa94-cd44801747fd None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:41.587640 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 100/187] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:41.593348 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305abbb0-bba7-4b23-9de5-13630adfb5b4 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:41.593905 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305abbb0-bba7-4b23-9de5-13630adfb5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:41.593905 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305abbb0-bba7-4b23-9de5-13630adfb5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:41.614620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-305abbb0-bba7-4b23-9de5-13630adfb5b4 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:41.615821 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-305abbb0-bba7-4b23-9de5-13630adfb5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:41.685387 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 88/188] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:41.698134 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af704f3-a24c-40bf-81e5-ce5b7023e7bf None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:41.698625 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af704f3-a24c-40bf-81e5-ce5b7023e7bf None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:41.698973 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af704f3-a24c-40bf-81e5-ce5b7023e7bf None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:41.700435 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 101/189] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:41.705619 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a80f9-4962-48f2-85bf-b3241fcf4a84 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:41.705907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a80f9-4962-48f2-85bf-b3241fcf4a84 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:41.706179 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a80f9-4962-48f2-85bf-b3241fcf4a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:41.724386 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-715a80f9-4962-48f2-85bf-b3241fcf4a84 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:41.725719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-715a80f9-4962-48f2-85bf-b3241fcf4a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:41.787074 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:41.795265 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 89/190] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:41.951628 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af704f3-a24c-40bf-81e5-ce5b7023e7bf None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:41.982057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:42.058909 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af704f3-a24c-40bf-81e5-ce5b7023e7bf None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:42.072110 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a80f9-4962-48f2-85bf-b3241fcf4a84 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:42.095411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7aea3e-bdfa-4a0d-bf30-bf8d14a9996f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:42.096156 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7aea3e-bdfa-4a0d-bf30-bf8d14a9996f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:42.096436 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7aea3e-bdfa-4a0d-bf30-bf8d14a9996f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:42.096848 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7aea3e-bdfa-4a0d-bf30-bf8d14a9996f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:42.107690 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7aea3e-bdfa-4a0d-bf30-bf8d14a9996f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:42.112721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-ac7aea3e-bdfa-4a0d-bf30-bf8d14a9996f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 05:51:42.112721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:42.112721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:42.112721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:42.112721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:42.112721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:42.112721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:42.112721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:42.112721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:42.112721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:42.112721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:42.112721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:42.112721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:42.112721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:42.112721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:42.112721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:42.113793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:42.113793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:42.113793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:42.113793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:42.113793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:42.113793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:42.113793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:42.113793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:42.113793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:42.113793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:42.113793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:42.113793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:42.113793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:42.113793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:42.113793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:42.113793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:42.114605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:42.114605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:42.114605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:42.114605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:42.114605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:42.114605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:42.114605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:42.114605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:42.114605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:42.114605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:42.114605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 05:51:42.114605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:51:42.114605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 102/191] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:51:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:51:42.119646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7aea3e-bdfa-4a0d-bf30-bf8d14a9996f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:42.178331 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a80f9-4962-48f2-85bf-b3241fcf4a84 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:42.210543 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:42.219917 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d065b01-36b2-4833-880d-0c6ce13a2879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:42.220565 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d065b01-36b2-4833-880d-0c6ce13a2879 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:42.220880 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d065b01-36b2-4833-880d-0c6ce13a2879 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:42.221151 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d065b01-36b2-4833-880d-0c6ce13a2879 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:42.222758 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d065b01-36b2-4833-880d-0c6ce13a2879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:42.224356 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d065b01-36b2-4833-880d-0c6ce13a2879 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:42.225973 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 90/192] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:51:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:42.229388 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7aea3e-bdfa-4a0d-bf30-bf8d14a9996f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:42.231261 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d065b01-36b2-4833-880d-0c6ce13a2879 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:42.267340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57758436-80a5-4af0-afcc-f6321bdac07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:42.267944 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57758436-80a5-4af0-afcc-f6321bdac07f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:42.268224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57758436-80a5-4af0-afcc-f6321bdac07f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:42.268483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57758436-80a5-4af0-afcc-f6321bdac07f None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:42.269750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57758436-80a5-4af0-afcc-f6321bdac07f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:42.270066 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57758436-80a5-4af0-afcc-f6321bdac07f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:42.271438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57758436-80a5-4af0-afcc-f6321bdac07f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:42.279845 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 103/193] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Oct 20 05:51:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:42.285281 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57758436-80a5-4af0-afcc-f6321bdac07f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:42.303746 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:42.342311 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d065b01-36b2-4833-880d-0c6ce13a2879 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:42.374999 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7594d-bfe4-4358-bd45-d86257419015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:42.375620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7594d-bfe4-4358-bd45-d86257419015 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:42.375889 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7594d-bfe4-4358-bd45-d86257419015 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:42.376160 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7594d-bfe4-4358-bd45-d86257419015 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:42.376834 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7594d-bfe4-4358-bd45-d86257419015 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:42.378777 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7594d-bfe4-4358-bd45-d86257419015 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:42.382567 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57758436-80a5-4af0-afcc-f6321bdac07f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:42.400117 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 91/194] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Oct 20 05:51:42 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 05:51:42.401159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffef37b-42c9-4bc3-bf46-c44e726726cb None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:42.401441 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffef37b-42c9-4bc3-bf46-c44e726726cb None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:42.401712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffef37b-42c9-4bc3-bf46-c44e726726cb None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:42.402941 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 92/195] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:42.408942 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38756f6f-9e8d-41b6-ac5b-d2dec86d0457 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:42.409227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38756f6f-9e8d-41b6-ac5b-d2dec86d0457 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:42.409483 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38756f6f-9e8d-41b6-ac5b-d2dec86d0457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:42.426782 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfab135-8ebd-4dfd-a3ac-e1cb12c2c23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:42.427441 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfab135-8ebd-4dfd-a3ac-e1cb12c2c23c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:42.427721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfab135-8ebd-4dfd-a3ac-e1cb12c2c23c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:42.427993 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfab135-8ebd-4dfd-a3ac-e1cb12c2c23c None admin] PATH_INFO: `/v3/users/f33949cbdd7f4cfe95ad6b33d0c5ebf7` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:42.438074 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-38756f6f-9e8d-41b6-ac5b-d2dec86d0457 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:42.439328 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-38756f6f-9e8d-41b6-ac5b-d2dec86d0457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:42.444982 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfab135-8ebd-4dfd-a3ac-e1cb12c2c23c None admin] RBAC: Authorizing `identity:get_user(user_id=f33949cbdd7f4cfe95ad6b33d0c5ebf7)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:42.446475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfab135-8ebd-4dfd-a3ac-e1cb12c2c23c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:42.448396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 104/196] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:42 2022] GET /identity/v3/users/f33949cbdd7f4cfe95ad6b33d0c5ebf7 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:42.454047 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfab135-8ebd-4dfd-a3ac-e1cb12c2c23c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:42.466552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:42.521421 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 93/197] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 05:51:42.570351 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfab135-8ebd-4dfd-a3ac-e1cb12c2c23c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:42.607305 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995aad9-aa34-4fc3-8e48-102f6c81febf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:42.607926 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995aad9-aa34-4fc3-8e48-102f6c81febf None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:42.608195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995aad9-aa34-4fc3-8e48-102f6c81febf None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:42.608461 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995aad9-aa34-4fc3-8e48-102f6c81febf None admin] PATH_INFO: `/v3/projects/e5ab74dfb5014b01bea8909bd1d64559` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:42.609612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995aad9-aa34-4fc3-8e48-102f6c81febf None admin] RBAC: Authorizing `identity:get_project(project_id=e5ab74dfb5014b01bea8909bd1d64559)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:42.611028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995aad9-aa34-4fc3-8e48-102f6c81febf None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:42.613535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 105/198] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:51:42 2022] GET /identity/v3/projects/e5ab74dfb5014b01bea8909bd1d64559 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:42.619088 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38756f6f-9e8d-41b6-ac5b-d2dec86d0457 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:42.676629 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:42.728148 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38756f6f-9e8d-41b6-ac5b-d2dec86d0457 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:42.764555 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c6483-2f6f-4071-b635-34c91a5068fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:42.765264 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c6483-2f6f-4071-b635-34c91a5068fc None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:42.765542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c6483-2f6f-4071-b635-34c91a5068fc None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:42.765812 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c6483-2f6f-4071-b635-34c91a5068fc None admin] PATH_INFO: `/v3/projects/e5ab74dfb5014b01bea8909bd1d64559/users/f33949cbdd7f4cfe95ad6b33d0c5ebf7/roles/78820dc7bd3841cfbc80e594e4e77171` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:42.767730 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c6483-2f6f-4071-b635-34c91a5068fc None admin] RBAC: Authorizing `identity:create_grant(project_id=e5ab74dfb5014b01bea8909bd1d64559, user_id=f33949cbdd7f4cfe95ad6b33d0c5ebf7, role_id=78820dc7bd3841cfbc80e594e4e77171)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:42.769218 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c6483-2f6f-4071-b635-34c91a5068fc None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:42.783626 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 94/199] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:51:42 2022] PUT /identity/v3/projects/e5ab74dfb5014b01bea8909bd1d64559/users/f33949cbdd7f4cfe95ad6b33d0c5ebf7/roles/78820dc7bd3841cfbc80e594e4e77171 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:51:42.793174 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995aad9-aa34-4fc3-8e48-102f6c81febf None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:42.891024 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:42.918997 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995aad9-aa34-4fc3-8e48-102f6c81febf None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:42.959597 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89107062-b41c-43b6-996f-61efa8d19769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:42.960485 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89107062-b41c-43b6-996f-61efa8d19769 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:42.960969 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89107062-b41c-43b6-996f-61efa8d19769 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:42.961393 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89107062-b41c-43b6-996f-61efa8d19769 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:42.973249 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89107062-b41c-43b6-996f-61efa8d19769 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:42.978948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-89107062-b41c-43b6-996f-61efa8d19769 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:51:42.978948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:42.978948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:42.978948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:42.978948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:42.978948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:42.978948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:42.978948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:42.978948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:42.978948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:42.978948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:42.978948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:42.978948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:42.978948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:42.978948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:42.978948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:42.979999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:42.979999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:42.979999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:42.979999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:42.979999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:42.979999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:42.979999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:42.979999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:42.979999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:42.979999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:42.979999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:42.979999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:42.979999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:42.979999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:42.979999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:42.979999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:42.981014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:42.981014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:42.981014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:42.981014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:42.981014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:42.981014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:42.981014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:42.981014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:42.981014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:42.981014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:42.981014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:51:42.981014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:51:42.982952 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 106/200] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:51:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:51:42.988996 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c6483-2f6f-4071-b635-34c91a5068fc None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:43.092066 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c6483-2f6f-4071-b635-34c91a5068fc None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:43.103433 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:43.127575 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f0900-920b-418c-86b6-a2b755db775b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:43.128188 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f0900-920b-418c-86b6-a2b755db775b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:43.129179 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f0900-920b-418c-86b6-a2b755db775b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:43.129451 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f0900-920b-418c-86b6-a2b755db775b None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:43.130119 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f0900-920b-418c-86b6-a2b755db775b None admin] RBAC: Adding query filter params (name=member) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:43.130419 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f0900-920b-418c-86b6-a2b755db775b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:43.131787 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f0900-920b-418c-86b6-a2b755db775b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:43.140835 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 95/201] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Oct 20 05:51:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:43.147600 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89107062-b41c-43b6-996f-61efa8d19769 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:43.259829 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89107062-b41c-43b6-996f-61efa8d19769 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:43.308447 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c99ff-aadb-440d-bf6a-d78df9fe4330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:43.309413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c99ff-aadb-440d-bf6a-d78df9fe4330 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:43.309884 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c99ff-aadb-440d-bf6a-d78df9fe4330 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:43.310343 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c99ff-aadb-440d-bf6a-d78df9fe4330 None admin] PATH_INFO: `/v3/users/f944919f7f08497dbad4b41cf82a240b` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:43.311718 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c99ff-aadb-440d-bf6a-d78df9fe4330 None admin] RBAC: Authorizing `identity:get_user(user_id=f944919f7f08497dbad4b41cf82a240b)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:43.313381 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c99ff-aadb-440d-bf6a-d78df9fe4330 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:43.319952 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 107/202] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:43 2022] GET /identity/v3/users/f944919f7f08497dbad4b41cf82a240b => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:43.321818 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f0900-920b-418c-86b6-a2b755db775b None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:43.430714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f0900-920b-418c-86b6-a2b755db775b None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:43.464363 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd503b-f615-4725-a523-3285ac6d4983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:43.465017 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd503b-f615-4725-a523-3285ac6d4983 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:43.465295 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd503b-f615-4725-a523-3285ac6d4983 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:43.465562 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd503b-f615-4725-a523-3285ac6d4983 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:43.513341 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:43.541017 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd503b-f615-4725-a523-3285ac6d4983 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:43.542477 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd503b-f615-4725-a523-3285ac6d4983 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:43.641173 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 96/203] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:51:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 05:51:43.646902 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c99ff-aadb-440d-bf6a-d78df9fe4330 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:43.651573 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:43.758529 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c99ff-aadb-440d-bf6a-d78df9fe4330 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:43.794783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2e91a-1473-4254-b79f-af097904e6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:43.795715 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2e91a-1473-4254-b79f-af097904e6c9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:43.796235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2e91a-1473-4254-b79f-af097904e6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:43.796718 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2e91a-1473-4254-b79f-af097904e6c9 None admin] PATH_INFO: `/v3/projects/e5ab74dfb5014b01bea8909bd1d64559` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:43.798010 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2e91a-1473-4254-b79f-af097904e6c9 None admin] RBAC: Authorizing `identity:get_project(project_id=e5ab74dfb5014b01bea8909bd1d64559)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:43.799746 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2e91a-1473-4254-b79f-af097904e6c9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:43.801965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 108/204] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:51:43 2022] GET /identity/v3/projects/e5ab74dfb5014b01bea8909bd1d64559 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:43.809730 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd503b-f615-4725-a523-3285ac6d4983 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:43.881176 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:43.912887 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd503b-f615-4725-a523-3285ac6d4983 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:43.948000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ff81f-2c45-4e10-8733-f6c643468b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:43.948605 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ff81f-2c45-4e10-8733-f6c643468b6d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:43.948916 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ff81f-2c45-4e10-8733-f6c643468b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:43.949181 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ff81f-2c45-4e10-8733-f6c643468b6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:43.949963 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ff81f-2c45-4e10-8733-f6c643468b6d None admin] RBAC: Adding query filter params (role.id=c1e0df53e1264ec79d91065e25acd29d, scope.project.id=e5ab74dfb5014b01bea8909bd1d64559, user.id=f944919f7f08497dbad4b41cf82a240b) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:43.950265 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ff81f-2c45-4e10-8733-f6c643468b6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:43.951740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ff81f-2c45-4e10-8733-f6c643468b6d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:43.958471 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 97/205] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:51:43 2022] GET /identity/v3/role_assignments?user.id=f944919f7f08497dbad4b41cf82a240b&scope.project.id=e5ab74dfb5014b01bea8909bd1d64559&role.id=c1e0df53e1264ec79d91065e25acd29d => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:44.779277 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556174b-618d-4327-8b74-524883bd0991 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:44.779810 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556174b-618d-4327-8b74-524883bd0991 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:44.779810 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556174b-618d-4327-8b74-524883bd0991 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:44.781375 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 109/206] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:44.786499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be40878-e695-4185-98a4-16cd3929adcf None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:44.786899 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be40878-e695-4185-98a4-16cd3929adcf None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:44.787073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be40878-e695-4185-98a4-16cd3929adcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:44.805793 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-6be40878-e695-4185-98a4-16cd3929adcf None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:44.806976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6be40878-e695-4185-98a4-16cd3929adcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:44.844178 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f93e1-6576-4582-93e7-e5b5f0110187 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:44.844449 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f93e1-6576-4582-93e7-e5b5f0110187 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:44.844742 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f93e1-6576-4582-93e7-e5b5f0110187 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:44.846080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 110/207] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:51:44.851933 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabe47d-76e2-44a4-bf8e-83d68ad00500 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:44.852216 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabe47d-76e2-44a4-bf8e-83d68ad00500 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:44.852486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabe47d-76e2-44a4-bf8e-83d68ad00500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:44.870419 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-1dabe47d-76e2-44a4-bf8e-83d68ad00500 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:44.871587 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1dabe47d-76e2-44a4-bf8e-83d68ad00500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:44.883692 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 98/208] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:44.947608 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 111/209] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 05:51:45.017969 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00661a38-a69b-48e8-9e22-619546c20ebc None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:45.018268 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00661a38-a69b-48e8-9e22-619546c20ebc None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:45.018521 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00661a38-a69b-48e8-9e22-619546c20ebc None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:45.020100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 99/210] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:45.025051 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4775c5-d599-4d05-8383-5e55fc1fad7c None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:45.025340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4775c5-d599-4d05-8383-5e55fc1fad7c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:45.025611 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4775c5-d599-4d05-8383-5e55fc1fad7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:45.031879 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:45.045643 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-1b4775c5-d599-4d05-8383-5e55fc1fad7c None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:45.046812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1b4775c5-d599-4d05-8383-5e55fc1fad7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:45.113833 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 112/211] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:45.153637 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00661a38-a69b-48e8-9e22-619546c20ebc None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:45.220865 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4775c5-d599-4d05-8383-5e55fc1fad7c None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:45.221893 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:45.260016 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00661a38-a69b-48e8-9e22-619546c20ebc None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:45.297258 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1f38f-5e3c-42d4-8631-805b6eaa2d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:45.297821 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1f38f-5e3c-42d4-8631-805b6eaa2d78 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:45.298098 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1f38f-5e3c-42d4-8631-805b6eaa2d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:45.298367 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1f38f-5e3c-42d4-8631-805b6eaa2d78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:45.299660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1f38f-5e3c-42d4-8631-805b6eaa2d78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:45.301497 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1f38f-5e3c-42d4-8631-805b6eaa2d78 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:45.307955 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 100/212] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:51:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:45.309015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1f38f-5e3c-42d4-8631-805b6eaa2d78 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:45.334627 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4775c5-d599-4d05-8383-5e55fc1fad7c None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:45.366482 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:45.371997 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7475d-5dbb-44a6-8d76-b987a8aa389b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:45.372628 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7475d-5dbb-44a6-8d76-b987a8aa389b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:45.372999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7475d-5dbb-44a6-8d76-b987a8aa389b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:45.373276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7475d-5dbb-44a6-8d76-b987a8aa389b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:45.384753 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7475d-5dbb-44a6-8d76-b987a8aa389b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:45.389887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-e4b7475d-5dbb-44a6-8d76-b987a8aa389b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:51:45.389887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:45.389887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:45.389887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:45.389887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:45.389887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:45.389887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:45.389887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:45.389887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:45.389887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:45.389887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:45.389887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:45.389887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:45.389887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:45.389887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:45.389887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:45.390843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:45.390843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:45.390843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:45.390843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:45.390843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:45.390843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:45.390843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:45.390843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:45.390843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:45.390843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:45.390843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:45.390843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:45.390843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:45.390843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:45.390843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:45.390843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:45.391625 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:45.391625 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:45.391625 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:45.391625 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:45.391625 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:45.391625 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:45.391625 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:45.391625 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:45.391625 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:45.391625 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:45.391625 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:51:45.391625 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:51:45.391625 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 113/213] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:51:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:51:45.392269 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7475d-5dbb-44a6-8d76-b987a8aa389b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:45.414392 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1f38f-5e3c-42d4-8631-805b6eaa2d78 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:45.449067 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87d3be-840d-48f3-8826-7d75e195fb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:45.449666 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87d3be-840d-48f3-8826-7d75e195fb6f None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:45.449939 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87d3be-840d-48f3-8826-7d75e195fb6f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:45.450205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87d3be-840d-48f3-8826-7d75e195fb6f None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:45.450842 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87d3be-840d-48f3-8826-7d75e195fb6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:45.452272 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87d3be-840d-48f3-8826-7d75e195fb6f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:45.460216 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:45.472988 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 101/214] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Oct 20 05:51:45 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:51:45.473777 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87d3be-840d-48f3-8826-7d75e195fb6f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:45.491510 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7475d-5dbb-44a6-8d76-b987a8aa389b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:45.530815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf86d1-7bd6-4f78-85fd-45cf4266c069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:45.531464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf86d1-7bd6-4f78-85fd-45cf4266c069 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:45.531741 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf86d1-7bd6-4f78-85fd-45cf4266c069 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:45.532009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf86d1-7bd6-4f78-85fd-45cf4266c069 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:45.544281 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf86d1-7bd6-4f78-85fd-45cf4266c069 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:45.549340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-cabf86d1-7bd6-4f78-85fd-45cf4266c069 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 05:51:45.549340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:45.549340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:45.549340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:45.549340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:45.549340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:45.549340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:45.549340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:45.549340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:45.549340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:45.549340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:45.549340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:45.549340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:45.549340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:45.549340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:45.549340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:45.550334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:45.550334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:45.550334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:45.550334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:45.550334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:45.550334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:45.550334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:45.550334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:45.550334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:45.550334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:45.550334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:45.550334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:45.550334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:45.550334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:45.550334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:45.550334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:45.551412 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:45.551412 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:45.551412 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:45.551412 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:45.551412 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:45.551412 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:45.551412 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:45.551412 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:45.551412 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:45.551412 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:45.551412 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 05:51:45.551412 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:51:45.551412 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 114/215] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:51:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:51:45.556455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf86d1-7bd6-4f78-85fd-45cf4266c069 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:45.603476 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87d3be-840d-48f3-8826-7d75e195fb6f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:45.614755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:45.643069 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab2dca-3c1f-4d73-a749-18ce61bd2a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:45.643696 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab2dca-3c1f-4d73-a749-18ce61bd2a37 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:45.644801 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab2dca-3c1f-4d73-a749-18ce61bd2a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:45.645083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab2dca-3c1f-4d73-a749-18ce61bd2a37 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:45.645759 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab2dca-3c1f-4d73-a749-18ce61bd2a37 None admin] RBAC: Adding query filter params (name=member) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:45.646067 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab2dca-3c1f-4d73-a749-18ce61bd2a37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:45.647454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab2dca-3c1f-4d73-a749-18ce61bd2a37 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:45.656441 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 102/216] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Oct 20 05:51:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:45.663199 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab2dca-3c1f-4d73-a749-18ce61bd2a37 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:45.676272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf86d1-7bd6-4f78-85fd-45cf4266c069 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:45.714135 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb35668-4568-4831-9a2c-960b379b49e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:45.714745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb35668-4568-4831-9a2c-960b379b49e4 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:45.715026 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb35668-4568-4831-9a2c-960b379b49e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:45.715295 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb35668-4568-4831-9a2c-960b379b49e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:45.715963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb35668-4568-4831-9a2c-960b379b49e4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:45.716255 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb35668-4568-4831-9a2c-960b379b49e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:45.717669 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb35668-4568-4831-9a2c-960b379b49e4 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:45.724423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:45.729004 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 115/217] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Oct 20 05:51:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:45.734830 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb35668-4568-4831-9a2c-960b379b49e4 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:45.787770 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab2dca-3c1f-4d73-a749-18ce61bd2a37 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:45.831031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f062c-1aac-43b3-874b-3865da44326c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:45.831210 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f062c-1aac-43b3-874b-3865da44326c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:45.831479 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f062c-1aac-43b3-874b-3865da44326c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:45.833084 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f062c-1aac-43b3-874b-3865da44326c None admin] PATH_INFO: `/v3/users/981510bbacba4df1ba9149db423ded9b` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:45.855192 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f062c-1aac-43b3-874b-3865da44326c None admin] RBAC: Authorizing `identity:get_user(user_id=981510bbacba4df1ba9149db423ded9b)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:45.856720 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f062c-1aac-43b3-874b-3865da44326c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:45.858430 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 103/218] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:45 2022] GET /identity/v3/users/981510bbacba4df1ba9149db423ded9b => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:45.864902 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f062c-1aac-43b3-874b-3865da44326c None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:45.873389 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb35668-4568-4831-9a2c-960b379b49e4 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:45.909084 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:45.921386 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a523c7-28a8-4b43-ba33-c03e2bd360dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:45.921955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a523c7-28a8-4b43-ba33-c03e2bd360dd None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:45.922286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a523c7-28a8-4b43-ba33-c03e2bd360dd None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:45.922566 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a523c7-28a8-4b43-ba33-c03e2bd360dd None admin] PATH_INFO: `/v3/users/f33949cbdd7f4cfe95ad6b33d0c5ebf7` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:45.924426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a523c7-28a8-4b43-ba33-c03e2bd360dd None admin] RBAC: Authorizing `identity:get_user(user_id=f33949cbdd7f4cfe95ad6b33d0c5ebf7)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:45.926136 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a523c7-28a8-4b43-ba33-c03e2bd360dd None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:45.928524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 116/219] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:45 2022] GET /identity/v3/users/f33949cbdd7f4cfe95ad6b33d0c5ebf7 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:45.934556 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a523c7-28a8-4b43-ba33-c03e2bd360dd None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:45.989318 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f062c-1aac-43b3-874b-3865da44326c None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:46.001955 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:46.027791 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb94f1-4039-4107-a931-ff42eb7173c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:46.028587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb94f1-4039-4107-a931-ff42eb7173c9 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:46.029001 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb94f1-4039-4107-a931-ff42eb7173c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:46.029361 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb94f1-4039-4107-a931-ff42eb7173c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:46.030190 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb94f1-4039-4107-a931-ff42eb7173c9 None admin] RBAC: Adding query filter params (role.id=c1e0df53e1264ec79d91065e25acd29d, user.id=981510bbacba4df1ba9149db423ded9b, scope.system=all) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:46.030589 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb94f1-4039-4107-a931-ff42eb7173c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:46.032172 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb94f1-4039-4107-a931-ff42eb7173c9 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:46.038083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 104/220] 10.222.0.20 () {56 vars in 1308 bytes} [Thu Oct 20 05:51:45 2022] GET /identity/v3/role_assignments?user.id=981510bbacba4df1ba9149db423ded9b&scope.system=all&role.id=c1e0df53e1264ec79d91065e25acd29d => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:46.070996 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a523c7-28a8-4b43-ba33-c03e2bd360dd None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:46.114661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2f100-4182-4500-a1db-34dbeb7a5c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:46.115204 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2f100-4182-4500-a1db-34dbeb7a5c47 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:46.115531 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2f100-4182-4500-a1db-34dbeb7a5c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:46.115806 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2f100-4182-4500-a1db-34dbeb7a5c47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:46.179740 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:46.201714 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2f100-4182-4500-a1db-34dbeb7a5c47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:46.203215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2f100-4182-4500-a1db-34dbeb7a5c47 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:46.319082 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 117/221] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:51:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 20 05:51:46.324279 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb94f1-4039-4107-a931-ff42eb7173c9 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:46.427804 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb94f1-4039-4107-a931-ff42eb7173c9 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:46.462858 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92154b-55d5-4fbf-bfd9-e0a789239c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:46.463493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92154b-55d5-4fbf-bfd9-e0a789239c22 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:46.463764 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92154b-55d5-4fbf-bfd9-e0a789239c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:46.464031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92154b-55d5-4fbf-bfd9-e0a789239c22 None admin] PATH_INFO: `/v3/projects/e5ab74dfb5014b01bea8909bd1d64559` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:46.465170 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc92154b-55d5-4fbf-bfd9-e0a789239c22 None admin] RBAC: Authorizing `identity:get_project(project_id=e5ab74dfb5014b01bea8909bd1d64559)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:46.466608 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc92154b-55d5-4fbf-bfd9-e0a789239c22 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:46.468521 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 105/222] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:51:46 2022] GET /identity/v3/projects/e5ab74dfb5014b01bea8909bd1d64559 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:46.474662 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2f100-4182-4500-a1db-34dbeb7a5c47 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:46.564827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:46.580206 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2f100-4182-4500-a1db-34dbeb7a5c47 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:46.617736 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07fceb-7961-47d6-a1cf-542450daada4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:46.618340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07fceb-7961-47d6-a1cf-542450daada4 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:46.618609 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07fceb-7961-47d6-a1cf-542450daada4 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:46.618884 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07fceb-7961-47d6-a1cf-542450daada4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:46.619728 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a07fceb-7961-47d6-a1cf-542450daada4 None admin] RBAC: Adding query filter params (role.id=78820dc7bd3841cfbc80e594e4e77171, scope.project.id=e5ab74dfb5014b01bea8909bd1d64559, user.id=f33949cbdd7f4cfe95ad6b33d0c5ebf7) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:46.620032 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a07fceb-7961-47d6-a1cf-542450daada4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:46.621576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a07fceb-7961-47d6-a1cf-542450daada4 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:46.627564 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 118/223] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:51:46 2022] GET /identity/v3/role_assignments?user.id=f33949cbdd7f4cfe95ad6b33d0c5ebf7&scope.project.id=e5ab74dfb5014b01bea8909bd1d64559&role.id=78820dc7bd3841cfbc80e594e4e77171 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:47.822821 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0fe7f-9d06-4cae-9984-a6b4fde57b8f None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:47.823345 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0fe7f-9d06-4cae-9984-a6b4fde57b8f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:47.823345 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0fe7f-9d06-4cae-9984-a6b4fde57b8f None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:47.824751 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 106/224] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:47.829815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f69e8-229e-462e-99ba-0e4c29c7a011 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:47.830221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f69e8-229e-462e-99ba-0e4c29c7a011 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:47.830381 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f69e8-229e-462e-99ba-0e4c29c7a011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:47.849214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-2f7f69e8-229e-462e-99ba-0e4c29c7a011 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:47.850396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2f7f69e8-229e-462e-99ba-0e4c29c7a011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:47.878840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a5c22-de74-4f89-ab5f-4cd0b9196303 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:47.879189 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a5c22-de74-4f89-ab5f-4cd0b9196303 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:47.879403 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a5c22-de74-4f89-ab5f-4cd0b9196303 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:47.881288 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 107/225] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:47.887146 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d94325-103f-4f07-aada-e7a1e929d97f None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:47.887621 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d94325-103f-4f07-aada-e7a1e929d97f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:47.888089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d94325-103f-4f07-aada-e7a1e929d97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:47.907987 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-01d94325-103f-4f07-aada-e7a1e929d97f None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:47.909224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-01d94325-103f-4f07-aada-e7a1e929d97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:47.923334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 119/226] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:47.946964 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:47.979578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 108/227] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 05:51:48.204141 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f69e8-229e-462e-99ba-0e4c29c7a011 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:48.241556 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f200555-e038-4d8e-b807-a215ff42f034 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:48.242199 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f200555-e038-4d8e-b807-a215ff42f034 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:48.242673 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f200555-e038-4d8e-b807-a215ff42f034 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:48.244228 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 109/228] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:51:48.247722 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f200555-e038-4d8e-b807-a215ff42f034 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:48.281659 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:48.308352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f69e8-229e-462e-99ba-0e4c29c7a011 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:48.341286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0402132-03f3-41e0-a980-c9a56e823371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:48.341912 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0402132-03f3-41e0-a980-c9a56e823371 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:48.342189 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0402132-03f3-41e0-a980-c9a56e823371 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:48.342455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0402132-03f3-41e0-a980-c9a56e823371 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:48.350331 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f200555-e038-4d8e-b807-a215ff42f034 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:48.352774 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0402132-03f3-41e0-a980-c9a56e823371 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:48.357603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-e0402132-03f3-41e0-a980-c9a56e823371 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 05:51:48.357603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:48.357603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:48.357603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:48.357603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:48.357603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:48.357603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:48.357603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:48.357603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:48.357603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:48.357603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:48.357603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:48.357603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:48.357603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:48.357603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:48.357603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:48.358455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:48.358455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:48.358455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:48.358455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:48.358455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:48.358455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:48.358455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:48.358455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:48.358455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:48.358455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:48.358455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:48.358455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:48.358455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:48.358455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:48.358455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:48.358455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:48.359251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:48.359251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:48.359251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:48.359251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:48.359251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:48.359251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:48.359251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:48.359251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:48.359251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:48.359251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:48.359251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 05:51:48.359251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:51:48.359251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 120/229] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:51:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:51:48.359902 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f817971-87ef-4aa3-8db6-c1d9f53e6ad8 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:48.360719 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f817971-87ef-4aa3-8db6-c1d9f53e6ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:48.360719 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f817971-87ef-4aa3-8db6-c1d9f53e6ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:48.378568 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-6f817971-87ef-4aa3-8db6-c1d9f53e6ad8 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:48.379816 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6f817971-87ef-4aa3-8db6-c1d9f53e6ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:48.385906 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644e61d-a092-4c60-a858-b7219379de15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:48.386543 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644e61d-a092-4c60-a858-b7219379de15 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:48.386824 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644e61d-a092-4c60-a858-b7219379de15 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:48.387100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644e61d-a092-4c60-a858-b7219379de15 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:48.388196 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0644e61d-a092-4c60-a858-b7219379de15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:48.389666 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0644e61d-a092-4c60-a858-b7219379de15 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:48.391235 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 110/230] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:51:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:48.392073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644e61d-a092-4c60-a858-b7219379de15 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:48.435200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:48.448383 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 121/231] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:48.452904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f817971-87ef-4aa3-8db6-c1d9f53e6ad8 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:48.504732 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644e61d-a092-4c60-a858-b7219379de15 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:48.515852 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:48.545121 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88bd73-54f2-4718-bb54-59eaf96a80db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:48.545718 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88bd73-54f2-4718-bb54-59eaf96a80db None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:48.545994 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88bd73-54f2-4718-bb54-59eaf96a80db None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:48.546274 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88bd73-54f2-4718-bb54-59eaf96a80db None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:48.546920 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a88bd73-54f2-4718-bb54-59eaf96a80db None admin] RBAC: Adding query filter params (name=reader) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:48.547368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a88bd73-54f2-4718-bb54-59eaf96a80db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:48.548927 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a88bd73-54f2-4718-bb54-59eaf96a80db None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:48.554199 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f817971-87ef-4aa3-8db6-c1d9f53e6ad8 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:48.561017 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 111/232] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Oct 20 05:51:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:48.564359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88bd73-54f2-4718-bb54-59eaf96a80db None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:48.591634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf2844-8602-45b1-9a15-cfac5cf49faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:48.592575 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf2844-8602-45b1-9a15-cfac5cf49faf None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:48.593090 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf2844-8602-45b1-9a15-cfac5cf49faf None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:48.593415 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf2844-8602-45b1-9a15-cfac5cf49faf None admin] PATH_INFO: `/v3/groups` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:48.594169 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bf2844-8602-45b1-9a15-cfac5cf49faf None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:48.595735 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bf2844-8602-45b1-9a15-cfac5cf49faf None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:48.613478 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 122/233] 10.222.0.20 () {60 vars in 1134 bytes} [Thu Oct 20 05:51:48 2022] POST /identity/v3/groups => generated 229 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:51:48.677060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88bd73-54f2-4718-bb54-59eaf96a80db None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:48.678895 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:48.716543 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf2844-8602-45b1-9a15-cfac5cf49faf None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:48.716756 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e49f61-9e26-43d1-bf9b-a333744d80f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:48.717281 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e49f61-9e26-43d1-bf9b-a333744d80f9 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:48.717563 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e49f61-9e26-43d1-bf9b-a333744d80f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:48.717838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e49f61-9e26-43d1-bf9b-a333744d80f9 None admin] PATH_INFO: `/v3/users/9de92c67be1546b7b383527b7ebaf191` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:48.733106 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e49f61-9e26-43d1-bf9b-a333744d80f9 None admin] RBAC: Authorizing `identity:get_user(user_id=9de92c67be1546b7b383527b7ebaf191)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:48.734615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e49f61-9e26-43d1-bf9b-a333744d80f9 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:48.736636 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 112/234] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:48 2022] GET /identity/v3/users/9de92c67be1546b7b383527b7ebaf191 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:48.742947 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e49f61-9e26-43d1-bf9b-a333744d80f9 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:48.843738 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf2844-8602-45b1-9a15-cfac5cf49faf None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:48.851752 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e49f61-9e26-43d1-bf9b-a333744d80f9 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:48.873544 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:48.885518 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e450f-ac25-46f9-83a6-bbf65699cb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:48.886200 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e450f-ac25-46f9-83a6-bbf65699cb57 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:48.886433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e450f-ac25-46f9-83a6-bbf65699cb57 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:48.886703 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e450f-ac25-46f9-83a6-bbf65699cb57 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:48.889005 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0fbdf-debd-4ccf-88d2-7fb13f74e48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:48.889614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0fbdf-debd-4ccf-88d2-7fb13f74e48b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:48.889888 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0fbdf-debd-4ccf-88d2-7fb13f74e48b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:48.890157 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0fbdf-debd-4ccf-88d2-7fb13f74e48b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:48.890906 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc0fbdf-debd-4ccf-88d2-7fb13f74e48b None admin] RBAC: Adding query filter params (role.id=78820dc7bd3841cfbc80e594e4e77171, user.id=9de92c67be1546b7b383527b7ebaf191, scope.system=all) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:48.891188 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc0fbdf-debd-4ccf-88d2-7fb13f74e48b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:48.892652 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc0fbdf-debd-4ccf-88d2-7fb13f74e48b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:48.897242 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e450f-ac25-46f9-83a6-bbf65699cb57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:48.897993 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 113/235] 10.222.0.20 () {56 vars in 1308 bytes} [Thu Oct 20 05:51:48 2022] GET /identity/v3/role_assignments?user.id=9de92c67be1546b7b383527b7ebaf191&scope.system=all&role.id=78820dc7bd3841cfbc80e594e4e77171 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:48.903138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-2f0e450f-ac25-46f9-83a6-bbf65699cb57 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:51:48.903138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:48.903138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:48.903138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:48.903138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:48.903138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:48.903138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:48.903138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:48.903138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:48.903138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:48.903138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:48.903138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:48.903138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:48.903138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:48.903138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:48.903138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:48.904060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:48.904060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:48.904060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:48.904060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:48.904060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:48.904060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:48.904060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:48.904060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:48.904060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:48.904060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:48.904060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:48.904060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:48.904060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:48.904060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:48.904060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:48.904060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:48.905144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:48.905144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:48.905144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:48.905144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:48.905144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:48.905144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:48.905144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:48.905144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:48.905144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:48.905144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:48.905144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:51:48.905144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:51:48.905144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 123/236] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:51:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:51:48.911042 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0fbdf-debd-4ccf-88d2-7fb13f74e48b None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:48.942037 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:49.026262 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0fbdf-debd-4ccf-88d2-7fb13f74e48b None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:49.064346 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e01094-2834-405e-bbc4-956548988790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:49.064993 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e01094-2834-405e-bbc4-956548988790 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:49.065319 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e01094-2834-405e-bbc4-956548988790 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:49.065592 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e01094-2834-405e-bbc4-956548988790 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:49.067117 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e01094-2834-405e-bbc4-956548988790 None admin] RBAC: Adding query filter params (name=member) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:49.067372 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e01094-2834-405e-bbc4-956548988790 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:49.068772 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e01094-2834-405e-bbc4-956548988790 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:49.078842 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 114/237] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Oct 20 05:51:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:49.084839 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e450f-ac25-46f9-83a6-bbf65699cb57 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:49.185554 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e450f-ac25-46f9-83a6-bbf65699cb57 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:49.219079 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a2e497-32f9-48c4-b169-79f4b5cf4da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:49.219706 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2e497-32f9-48c4-b169-79f4b5cf4da9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:49.219980 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2e497-32f9-48c4-b169-79f4b5cf4da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:49.220248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2e497-32f9-48c4-b169-79f4b5cf4da9 None admin] PATH_INFO: `/v3/users/981510bbacba4df1ba9149db423ded9b` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:49.221376 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a2e497-32f9-48c4-b169-79f4b5cf4da9 None admin] RBAC: Authorizing `identity:get_user(user_id=981510bbacba4df1ba9149db423ded9b)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:49.222977 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a2e497-32f9-48c4-b169-79f4b5cf4da9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:49.224737 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 124/238] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:49 2022] GET /identity/v3/users/981510bbacba4df1ba9149db423ded9b => generated 293 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:49.230084 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e01094-2834-405e-bbc4-956548988790 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:49.324865 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:49.340561 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e01094-2834-405e-bbc4-956548988790 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:49.375872 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0a06b-5384-4d0d-945b-9eacf7cfe4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:49.376530 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0a06b-5384-4d0d-945b-9eacf7cfe4e7 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:49.376857 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0a06b-5384-4d0d-945b-9eacf7cfe4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:49.377131 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0a06b-5384-4d0d-945b-9eacf7cfe4e7 None admin] PATH_INFO: `/v3/system/users/981510bbacba4df1ba9149db423ded9b/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:49.379140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e0a06b-5384-4d0d-945b-9eacf7cfe4e7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=981510bbacba4df1ba9149db423ded9b, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:49.380531 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e0a06b-5384-4d0d-945b-9eacf7cfe4e7 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:49.392876 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 115/239] 10.222.0.20 () {58 vars in 1334 bytes} [Thu Oct 20 05:51:49 2022] PUT /identity/v3/system/users/981510bbacba4df1ba9149db423ded9b/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:51:50.828684 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad949bd-ff57-42fd-9291-6dd5edf5f87d None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:50.829216 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad949bd-ff57-42fd-9291-6dd5edf5f87d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:50.829293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad949bd-ff57-42fd-9291-6dd5edf5f87d None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:50.830741 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 125/240] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:50.836507 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be6e8e-eb9a-4f19-b8b6-35d946044ca5 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:50.836916 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be6e8e-eb9a-4f19-b8b6-35d946044ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:50.837098 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be6e8e-eb9a-4f19-b8b6-35d946044ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:50.856277 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-34be6e8e-eb9a-4f19-b8b6-35d946044ca5 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:50.857498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-34be6e8e-eb9a-4f19-b8b6-35d946044ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:50.925662 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 116/241] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:51.038067 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11f9b2-d209-458b-b6bf-0ae9a98f8c80 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:51.038345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11f9b2-d209-458b-b6bf-0ae9a98f8c80 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:51.038775 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11f9b2-d209-458b-b6bf-0ae9a98f8c80 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:51.040120 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 126/242] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:51.045573 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f5cc4-62d7-4185-b295-0b95a8185392 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:51.045861 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f5cc4-62d7-4185-b295-0b95a8185392 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:51.046130 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f5cc4-62d7-4185-b295-0b95a8185392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:51.064681 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-cb3f5cc4-62d7-4185-b295-0b95a8185392 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:51.066868 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-cb3f5cc4-62d7-4185-b295-0b95a8185392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:51.137586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 117/243] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:51.197376 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df11f9b2-d209-458b-b6bf-0ae9a98f8c80 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:51.257023 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:51.310348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df11f9b2-d209-458b-b6bf-0ae9a98f8c80 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:51.350335 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52468256-dbca-4f75-81e5-080cb670763f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:51.350971 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52468256-dbca-4f75-81e5-080cb670763f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:51.351251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52468256-dbca-4f75-81e5-080cb670763f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:51.351525 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52468256-dbca-4f75-81e5-080cb670763f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:51.352618 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52468256-dbca-4f75-81e5-080cb670763f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:51.354090 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52468256-dbca-4f75-81e5-080cb670763f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:51.355832 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 127/244] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:51:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:51.361499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f5cc4-62d7-4185-b295-0b95a8185392 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:51.373527 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:51.419086 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52468256-dbca-4f75-81e5-080cb670763f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:51.477708 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f5cc4-62d7-4185-b295-0b95a8185392 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:51.520247 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c49dc-c5f2-4466-a297-959dc97efee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:51.520892 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c49dc-c5f2-4466-a297-959dc97efee5 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:51.521168 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c49dc-c5f2-4466-a297-959dc97efee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:51.521435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c49dc-c5f2-4466-a297-959dc97efee5 None admin] PATH_INFO: `/v3/groups` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:51.522069 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c49dc-c5f2-4466-a297-959dc97efee5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:51.522519 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52468256-dbca-4f75-81e5-080cb670763f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:51.523514 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c49dc-c5f2-4466-a297-959dc97efee5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:51.539278 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 118/245] 10.222.0.20 () {60 vars in 1134 bytes} [Thu Oct 20 05:51:51 2022] POST /identity/v3/groups => generated 226 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:51:51.540486 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ebe88-2321-43e2-b902-76ba70ef119b None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:51.540798 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ebe88-2321-43e2-b902-76ba70ef119b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:51.541075 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ebe88-2321-43e2-b902-76ba70ef119b None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:51.542279 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 119/246] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:51.547993 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516030c-92ab-4323-a05e-e038203540b9 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:51.548487 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516030c-92ab-4323-a05e-e038203540b9 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:51.548927 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516030c-92ab-4323-a05e-e038203540b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:51.554110 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:51.567781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf65d47-29a7-4048-a6a5-57296596b037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:51.568440 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf65d47-29a7-4048-a6a5-57296596b037 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:51.568749 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf65d47-29a7-4048-a6a5-57296596b037 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:51.569026 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf65d47-29a7-4048-a6a5-57296596b037 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:51.571254 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-1516030c-92ab-4323-a05e-e038203540b9 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:51.572862 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1516030c-92ab-4323-a05e-e038203540b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:51.580542 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf65d47-29a7-4048-a6a5-57296596b037 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:51.586904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-acf65d47-29a7-4048-a6a5-57296596b037 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 05:51:51.586904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:51.586904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:51.586904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:51.586904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:51.586904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:51.586904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:51.586904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:51.586904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:51.586904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:51.586904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:51.586904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:51.586904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:51.586904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:51.586904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:51.586904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:51.588050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:51.588050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:51.588050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:51.588050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:51.588050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:51.588050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:51.588050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:51.588050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:51.588050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:51.588050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:51.588050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:51.588050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:51.588050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:51.588050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:51.588050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:51.588050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:51.588986 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:51.588986 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:51.588986 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:51.588986 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:51.588986 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:51.588986 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:51.588986 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:51.588986 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:51.588986 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:51.588986 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:51.588986 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 05:51:51.588986 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:51:51.588986 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 128/247] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:51:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:51:51.594030 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf65d47-29a7-4048-a6a5-57296596b037 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:51.653946 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:51.665151 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 120/248] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 05:51:51.707531 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf65d47-29a7-4048-a6a5-57296596b037 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:51.742844 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3867bb2-3d54-42d2-bf60-6c4ca7fd866b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:51.743457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3867bb2-3d54-42d2-bf60-6c4ca7fd866b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:51.743737 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3867bb2-3d54-42d2-bf60-6c4ca7fd866b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:51.744007 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3867bb2-3d54-42d2-bf60-6c4ca7fd866b None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:51.744674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3867bb2-3d54-42d2-bf60-6c4ca7fd866b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:51.745026 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3867bb2-3d54-42d2-bf60-6c4ca7fd866b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:51.746401 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3867bb2-3d54-42d2-bf60-6c4ca7fd866b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:51.755660 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 129/249] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Oct 20 05:51:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:51.761439 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516030c-92ab-4323-a05e-e038203540b9 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:51.872614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516030c-92ab-4323-a05e-e038203540b9 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:51.913249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ce312-2484-4326-836d-1b9aa1f8b98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:51.913902 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ce312-2484-4326-836d-1b9aa1f8b98d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:51.914174 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ce312-2484-4326-836d-1b9aa1f8b98d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:51.914448 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ce312-2484-4326-836d-1b9aa1f8b98d None admin] PATH_INFO: `/v3/users/9de92c67be1546b7b383527b7ebaf191` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:51.915577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ce312-2484-4326-836d-1b9aa1f8b98d None admin] RBAC: Authorizing `identity:get_user(user_id=9de92c67be1546b7b383527b7ebaf191)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:51.917058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ce312-2484-4326-836d-1b9aa1f8b98d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:51.918990 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 121/250] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:51 2022] GET /identity/v3/users/9de92c67be1546b7b383527b7ebaf191 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:51.926175 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3867bb2-3d54-42d2-bf60-6c4ca7fd866b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:51.937967 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ce312-2484-4326-836d-1b9aa1f8b98d None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:51.988674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:52.023839 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:52.058391 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3867bb2-3d54-42d2-bf60-6c4ca7fd866b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:52.069586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ce312-2484-4326-836d-1b9aa1f8b98d None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:52.096819 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c2882-25de-45b3-9f7f-9f022c2443d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:52.098024 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c2882-25de-45b3-9f7f-9f022c2443d8 None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:52.098379 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c2882-25de-45b3-9f7f-9f022c2443d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:52.098824 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c2882-25de-45b3-9f7f-9f022c2443d8 None admin] PATH_INFO: `/v3/system/users/9de92c67be1546b7b383527b7ebaf191/roles/78820dc7bd3841cfbc80e594e4e77171` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:52.100257 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c2882-25de-45b3-9f7f-9f022c2443d8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9de92c67be1546b7b383527b7ebaf191, role_id=78820dc7bd3841cfbc80e594e4e77171)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:52.101928 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c2882-25de-45b3-9f7f-9f022c2443d8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:52.111620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ea190-d7f4-463f-85e6-e374f9dc3adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:52.112399 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ea190-d7f4-463f-85e6-e374f9dc3adc None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:52.113489 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ea190-d7f4-463f-85e6-e374f9dc3adc None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:52.113723 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ea190-d7f4-463f-85e6-e374f9dc3adc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:52.116117 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 130/251] 10.222.0.20 () {58 vars in 1334 bytes} [Thu Oct 20 05:51:51 2022] PUT /identity/v3/system/users/9de92c67be1546b7b383527b7ebaf191/roles/78820dc7bd3841cfbc80e594e4e77171 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:51:52.128705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5ea190-d7f4-463f-85e6-e374f9dc3adc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:52.134057 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-7c5ea190-d7f4-463f-85e6-e374f9dc3adc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:51:52.134057 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:52.134057 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:52.134057 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:52.134057 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:52.134057 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:52.134057 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:52.134057 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:52.134057 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:52.134057 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:52.134057 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:52.134057 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:52.134057 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:52.134057 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:52.134057 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:52.134057 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:52.135248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:52.135248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:52.135248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:52.135248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:52.135248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:52.135248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:52.135248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:52.135248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:52.135248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:52.135248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:52.135248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:52.135248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:52.135248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:52.135248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:52.135248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:52.135248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:52.136021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:52.136021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:52.136021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:52.136021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:52.136021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:52.136021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:52.136021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:52.136021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:52.136021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:52.136021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:52.136021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:51:52.136021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:51:52.136021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 122/252] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:51:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:51:52.142821 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c2882-25de-45b3-9f7f-9f022c2443d8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:52.265764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c2882-25de-45b3-9f7f-9f022c2443d8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:52.277348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:52.311435 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c83a7-f29b-4fc0-988f-53ac33546f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:52.312068 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c83a7-f29b-4fc0-988f-53ac33546f9c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:52.312348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c83a7-f29b-4fc0-988f-53ac33546f9c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:52.312629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c83a7-f29b-4fc0-988f-53ac33546f9c None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:52.313340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c83a7-f29b-4fc0-988f-53ac33546f9c None admin] RBAC: Adding query filter params (name=member) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:52.313647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c83a7-f29b-4fc0-988f-53ac33546f9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:52.315058 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c83a7-f29b-4fc0-988f-53ac33546f9c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:52.326230 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 131/253] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Oct 20 05:51:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:52.333782 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ea190-d7f4-463f-85e6-e374f9dc3adc None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:52.441267 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ea190-d7f4-463f-85e6-e374f9dc3adc None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:52.468994 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:52.477304 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa6cc6-0988-47ab-a7e1-8cec3d9e6f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:52.478436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa6cc6-0988-47ab-a7e1-8cec3d9e6f4c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:52.479070 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa6cc6-0988-47ab-a7e1-8cec3d9e6f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:52.479599 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa6cc6-0988-47ab-a7e1-8cec3d9e6f4c None admin] PATH_INFO: `/v3/users/981510bbacba4df1ba9149db423ded9b` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:52.481153 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaa6cc6-0988-47ab-a7e1-8cec3d9e6f4c None admin] RBAC: Authorizing `identity:get_user(user_id=981510bbacba4df1ba9149db423ded9b)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:52.482978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaa6cc6-0988-47ab-a7e1-8cec3d9e6f4c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:52.485014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 123/254] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:52 2022] GET /identity/v3/users/981510bbacba4df1ba9149db423ded9b => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:52.493804 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c83a7-f29b-4fc0-988f-53ac33546f9c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:52.632897 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c83a7-f29b-4fc0-988f-53ac33546f9c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:52.679257 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e24528-5525-4c39-99a5-c81ef114594a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:52.679881 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e24528-5525-4c39-99a5-c81ef114594a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:52.680162 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e24528-5525-4c39-99a5-c81ef114594a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:52.680437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e24528-5525-4c39-99a5-c81ef114594a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:52.681194 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e24528-5525-4c39-99a5-c81ef114594a None admin] RBAC: Adding query filter params (role.id=c1e0df53e1264ec79d91065e25acd29d, user.id=981510bbacba4df1ba9149db423ded9b, scope.system=all) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:52.681510 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e24528-5525-4c39-99a5-c81ef114594a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:52.682966 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e24528-5525-4c39-99a5-c81ef114594a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:52.701934 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 132/255] 10.222.0.20 () {56 vars in 1308 bytes} [Thu Oct 20 05:51:52 2022] GET /identity/v3/role_assignments?user.id=981510bbacba4df1ba9149db423ded9b&scope.system=all&role.id=c1e0df53e1264ec79d91065e25acd29d => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:54.629717 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1e501-2108-4928-8e3c-1facba7a5fcd None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:54.630312 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1e501-2108-4928-8e3c-1facba7a5fcd None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:54.630312 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1e501-2108-4928-8e3c-1facba7a5fcd None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:54.631573 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 124/256] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:54.637550 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625fff04-065b-421f-9f89-6d44d157f1e3 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:54.638026 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625fff04-065b-421f-9f89-6d44d157f1e3 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:54.638134 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625fff04-065b-421f-9f89-6d44d157f1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:54.676682 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-625fff04-065b-421f-9f89-6d44d157f1e3 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:54.677925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-625fff04-065b-421f-9f89-6d44d157f1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:54.714483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:54.755182 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 133/257] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 20 05:51:54.777980 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b8360-a47b-4c73-99ee-b0a1ef534be8 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:54.778647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b8360-a47b-4c73-99ee-b0a1ef534be8 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:54.779133 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b8360-a47b-4c73-99ee-b0a1ef534be8 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:54.780765 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 125/258] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:54.786952 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5b61c-7253-4acc-95a1-15baf28bd457 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:54.787257 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5b61c-7253-4acc-95a1-15baf28bd457 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:54.787514 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5b61c-7253-4acc-95a1-15baf28bd457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:54.813759 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-ced5b61c-7253-4acc-95a1-15baf28bd457 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:54.814970 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ced5b61c-7253-4acc-95a1-15baf28bd457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:54.826902 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 134/259] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 20 05:51:54.898676 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3ed4f-642f-4f8e-8e86-0c27ce261a72 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:54.900541 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3ed4f-642f-4f8e-8e86-0c27ce261a72 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:54.900541 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3ed4f-642f-4f8e-8e86-0c27ce261a72 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:54.900541 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 126/260] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:54.906112 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98baaa3-4595-4907-b43b-2a4770232a9d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:54.906398 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98baaa3-4595-4907-b43b-2a4770232a9d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:54.906672 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98baaa3-4595-4907-b43b-2a4770232a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:54.933640 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-e98baaa3-4595-4907-b43b-2a4770232a9d None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:54.934959 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e98baaa3-4595-4907-b43b-2a4770232a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:55.025245 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 135/261] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:55.120116 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3ed4f-642f-4f8e-8e86-0c27ce261a72 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:55.139134 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3ed4f-642f-4f8e-8e86-0c27ce261a72 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:55.144538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4af49-1a87-4fbf-9ba7-a9eac7c8ff33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:55.145424 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4af49-1a87-4fbf-9ba7-a9eac7c8ff33 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:55.145897 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4af49-1a87-4fbf-9ba7-a9eac7c8ff33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:55.146359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4af49-1a87-4fbf-9ba7-a9eac7c8ff33 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:55.164152 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4af49-1a87-4fbf-9ba7-a9eac7c8ff33 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:55.169394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-60a4af49-1a87-4fbf-9ba7-a9eac7c8ff33 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:51:55.169394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:55.169394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:55.169394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:55.169394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:55.169394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:55.169394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:55.169394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:55.169394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:55.169394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:55.169394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:55.169394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:55.169394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:55.169394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:55.169394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:55.169394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:55.170286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:55.170286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:55.170286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:55.170286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:55.170286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:55.170286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:55.170286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:55.170286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:55.170286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:55.170286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:55.170286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:55.170286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:55.170286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:55.170286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:55.170286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:55.170286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:55.171069 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:55.171069 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:55.171069 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:55.171069 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:55.171069 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:55.171069 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:55.171069 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:55.171069 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:55.171069 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:55.171069 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:55.171069 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:51:55.171069 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:51:55.173404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 127/262] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Oct 20 05:51:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 54 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:51:55.179931 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98baaa3-4595-4907-b43b-2a4770232a9d None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:55.200195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98baaa3-4595-4907-b43b-2a4770232a9d None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:55.214255 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad487a9-247c-4d1d-bc2a-1a80a34ba919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:55.214881 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad487a9-247c-4d1d-bc2a-1a80a34ba919 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:55.215158 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad487a9-247c-4d1d-bc2a-1a80a34ba919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:55.215427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad487a9-247c-4d1d-bc2a-1a80a34ba919 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:55.216097 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad487a9-247c-4d1d-bc2a-1a80a34ba919 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:55.216396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad487a9-247c-4d1d-bc2a-1a80a34ba919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:55.218210 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: WARNING py.warnings [None req-fad487a9-247c-4d1d-bc2a-1a80a34ba919 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 20 05:51:55.218210 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: warnings.warn(msg) Oct 20 05:51:55.218210 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: Oct 20 05:51:55.219106 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad487a9-247c-4d1d-bc2a-1a80a34ba919 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:55.227191 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4af49-1a87-4fbf-9ba7-a9eac7c8ff33 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:55.233092 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 136/263] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Oct 20 05:51:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:55.238541 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4af49-1a87-4fbf-9ba7-a9eac7c8ff33 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:55.239097 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad487a9-247c-4d1d-bc2a-1a80a34ba919 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:55.243596 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c6029-1750-4e48-9fce-a37e4c5fe964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:55.244482 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c6029-1750-4e48-9fce-a37e4c5fe964 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:55.245784 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c6029-1750-4e48-9fce-a37e4c5fe964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:55.246252 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c6029-1750-4e48-9fce-a37e4c5fe964 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:55.253639 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad487a9-247c-4d1d-bc2a-1a80a34ba919 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:55.258148 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c6029-1750-4e48-9fce-a37e4c5fe964 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:55.263566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-790c6029-1750-4e48-9fce-a37e4c5fe964 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 05:51:55.263566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:55.263566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:55.263566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:55.263566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:55.263566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:55.263566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:55.263566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:55.263566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:55.263566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:55.263566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:55.263566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:55.263566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:55.263566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:55.263566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:55.263566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:55.264417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:55.264417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:55.264417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:55.264417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:55.264417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:55.264417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:55.264417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:55.264417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:55.264417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:55.264417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:55.264417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:55.264417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:55.264417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:55.264417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:55.264417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:55.264417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:55.265550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:55.265550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:55.265550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:55.265550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:55.265550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:55.265550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:55.265550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:55.265550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:55.265550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:55.265550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:55.265550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 05:51:55.265550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:51:55.266152 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b1444-578a-4421-98b4-a54bc9eb1da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:55.266152 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b1444-578a-4421-98b4-a54bc9eb1da6 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:55.266152 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b1444-578a-4421-98b4-a54bc9eb1da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:55.266152 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b1444-578a-4421-98b4-a54bc9eb1da6 admin admin] PATH_INFO: `/v3/groups/8cdaa84a20c045fe900b1d8633f92fc4` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:55.268286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 128/264] 10.222.0.20 () {56 vars in 1135 bytes} [Thu Oct 20 05:51:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 42 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:51:55.272247 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b1444-578a-4421-98b4-a54bc9eb1da6 admin admin] RBAC: Authorizing `identity:get_group(group_id=8cdaa84a20c045fe900b1d8633f92fc4)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:55.273307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: WARNING py.warnings [None req-cd5b1444-578a-4421-98b4-a54bc9eb1da6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 05:51:55.273307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: warnings.warn(msg) Oct 20 05:51:55.273307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: Oct 20 05:51:55.275589 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c6029-1750-4e48-9fce-a37e4c5fe964 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:55.277519 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b1444-578a-4421-98b4-a54bc9eb1da6 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:55.279185 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 137/265] 10.222.0.20 () {56 vars in 1201 bytes} [Thu Oct 20 05:51:55 2022] GET /identity/v3/groups/8cdaa84a20c045fe900b1d8633f92fc4 => generated 226 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:55.284883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b1444-578a-4421-98b4-a54bc9eb1da6 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:55.287523 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c6029-1750-4e48-9fce-a37e4c5fe964 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:55.292205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99d5f4-9fb8-4ba5-a92f-cbc1f5e9f8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:55.293238 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99d5f4-9fb8-4ba5-a92f-cbc1f5e9f8b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:55.293768 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99d5f4-9fb8-4ba5-a92f-cbc1f5e9f8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:55.294282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99d5f4-9fb8-4ba5-a92f-cbc1f5e9f8b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:55.295154 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99d5f4-9fb8-4ba5-a92f-cbc1f5e9f8b7 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:55.295647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99d5f4-9fb8-4ba5-a92f-cbc1f5e9f8b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:55.297662 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: WARNING py.warnings [None req-db99d5f4-9fb8-4ba5-a92f-cbc1f5e9f8b7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 20 05:51:55.297662 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: warnings.warn(msg) Oct 20 05:51:55.297662 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: Oct 20 05:51:55.298882 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99d5f4-9fb8-4ba5-a92f-cbc1f5e9f8b7 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:55.313070 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b1444-578a-4421-98b4-a54bc9eb1da6 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:55.317205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 129/266] 10.222.0.20 () {56 vars in 1132 bytes} [Thu Oct 20 05:51:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:55.322017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af8367-5b22-4a34-8e9c-6d91b0a000c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:55.322603 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99d5f4-9fb8-4ba5-a92f-cbc1f5e9f8b7 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:55.322730 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af8367-5b22-4a34-8e9c-6d91b0a000c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:55.322912 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af8367-5b22-4a34-8e9c-6d91b0a000c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:55.323285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af8367-5b22-4a34-8e9c-6d91b0a000c8 admin admin] PATH_INFO: `/v3/projects/e78108de05fe409aa6e56461abb56cc0` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:55.324411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25af8367-5b22-4a34-8e9c-6d91b0a000c8 admin admin] RBAC: Authorizing `identity:get_project(project_id=e78108de05fe409aa6e56461abb56cc0)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:55.326488 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25af8367-5b22-4a34-8e9c-6d91b0a000c8 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:55.328235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 138/267] 10.222.0.20 () {56 vars in 1207 bytes} [Thu Oct 20 05:51:55 2022] GET /identity/v3/projects/e78108de05fe409aa6e56461abb56cc0 => generated 298 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:55.333887 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99d5f4-9fb8-4ba5-a92f-cbc1f5e9f8b7 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:55.334899 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af8367-5b22-4a34-8e9c-6d91b0a000c8 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:55.338690 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e55a81-3ea5-4d3f-9f4a-33886888e0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:55.339317 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e55a81-3ea5-4d3f-9f4a-33886888e0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:55.339647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e55a81-3ea5-4d3f-9f4a-33886888e0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:55.339933 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e55a81-3ea5-4d3f-9f4a-33886888e0d7 admin admin] PATH_INFO: `/v3/groups/8cdaa84a20c045fe900b1d8633f92fc4` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:55.341341 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e55a81-3ea5-4d3f-9f4a-33886888e0d7 admin admin] RBAC: Authorizing `identity:get_group(group_id=8cdaa84a20c045fe900b1d8633f92fc4)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:55.342476 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: WARNING py.warnings [None req-41e55a81-3ea5-4d3f-9f4a-33886888e0d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 05:51:55.342476 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: warnings.warn(msg) Oct 20 05:51:55.342476 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: Oct 20 05:51:55.343785 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e55a81-3ea5-4d3f-9f4a-33886888e0d7 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:55.345620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 130/268] 10.222.0.20 () {56 vars in 1201 bytes} [Thu Oct 20 05:51:55 2022] GET /identity/v3/groups/8cdaa84a20c045fe900b1d8633f92fc4 => generated 226 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:51:55.349084 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af8367-5b22-4a34-8e9c-6d91b0a000c8 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:55.353806 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e55a81-3ea5-4d3f-9f4a-33886888e0d7 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:55.360408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed1973-77f6-4f7f-ba63-a088e6946693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:55.363078 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed1973-77f6-4f7f-ba63-a088e6946693 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:55.363579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed1973-77f6-4f7f-ba63-a088e6946693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:55.364577 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed1973-77f6-4f7f-ba63-a088e6946693 admin admin] PATH_INFO: `/v3/projects/e78108de05fe409aa6e56461abb56cc0/groups/8cdaa84a20c045fe900b1d8633f92fc4/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:55.368574 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed1973-77f6-4f7f-ba63-a088e6946693 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e78108de05fe409aa6e56461abb56cc0, group_id=8cdaa84a20c045fe900b1d8633f92fc4, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:55.370990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: WARNING py.warnings [None req-65ed1973-77f6-4f7f-ba63-a088e6946693 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 05:51:55.370990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: warnings.warn(msg) Oct 20 05:51:55.370990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: Oct 20 05:51:55.374799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e55a81-3ea5-4d3f-9f4a-33886888e0d7 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:55.375970 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed1973-77f6-4f7f-ba63-a088e6946693 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:55.385168 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766b62b-2739-4f02-b357-9e86cd33417e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:55.385958 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766b62b-2739-4f02-b357-9e86cd33417e admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:55.386285 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766b62b-2739-4f02-b357-9e86cd33417e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:55.386881 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766b62b-2739-4f02-b357-9e86cd33417e admin admin] PATH_INFO: `/v3/projects/e78108de05fe409aa6e56461abb56cc0` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:55.388140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766b62b-2739-4f02-b357-9e86cd33417e admin admin] RBAC: Authorizing `identity:get_project(project_id=e78108de05fe409aa6e56461abb56cc0)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:55.390510 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766b62b-2739-4f02-b357-9e86cd33417e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:55.397460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 131/269] 10.222.0.20 () {56 vars in 1207 bytes} [Thu Oct 20 05:51:55 2022] GET /identity/v3/projects/e78108de05fe409aa6e56461abb56cc0 => generated 298 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:55.405677 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 139/270] 10.222.0.20 () {58 vars in 1463 bytes} [Thu Oct 20 05:51:55 2022] PUT /identity/v3/projects/e78108de05fe409aa6e56461abb56cc0/groups/8cdaa84a20c045fe900b1d8633f92fc4/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:51:55.407993 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766b62b-2739-4f02-b357-9e86cd33417e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:55.419477 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766b62b-2739-4f02-b357-9e86cd33417e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:55.498361 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1cd3a-1c0c-4fd5-a49b-da06bed749de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:55.499041 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1cd3a-1c0c-4fd5-a49b-da06bed749de admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:55.499321 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1cd3a-1c0c-4fd5-a49b-da06bed749de admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:55.499595 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1cd3a-1c0c-4fd5-a49b-da06bed749de admin admin] PATH_INFO: `/v3/projects/e78108de05fe409aa6e56461abb56cc0/groups/8cdaa84a20c045fe900b1d8633f92fc4/roles/49dae0405c764c96b7bfbcf9d9efd011` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:55.501939 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1cd3a-1c0c-4fd5-a49b-da06bed749de admin admin] RBAC: Authorizing `identity:create_grant(project_id=e78108de05fe409aa6e56461abb56cc0, group_id=8cdaa84a20c045fe900b1d8633f92fc4, role_id=49dae0405c764c96b7bfbcf9d9efd011)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:55.502973 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: WARNING py.warnings [None req-22e1cd3a-1c0c-4fd5-a49b-da06bed749de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 05:51:55.502973 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: warnings.warn(msg) Oct 20 05:51:55.502973 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: Oct 20 05:51:55.505222 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1cd3a-1c0c-4fd5-a49b-da06bed749de admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:55.529686 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 132/271] 10.222.0.20 () {58 vars in 1463 bytes} [Thu Oct 20 05:51:55 2022] PUT /identity/v3/projects/e78108de05fe409aa6e56461abb56cc0/groups/8cdaa84a20c045fe900b1d8633f92fc4/roles/49dae0405c764c96b7bfbcf9d9efd011 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 05:51:55.544867 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed1973-77f6-4f7f-ba63-a088e6946693 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:55.624508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:55.679883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed1973-77f6-4f7f-ba63-a088e6946693 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:55.725077 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1bf6b-a9f3-4037-8829-d12565545877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:55.725795 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1bf6b-a9f3-4037-8829-d12565545877 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:55.726077 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1bf6b-a9f3-4037-8829-d12565545877 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:55.726351 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1bf6b-a9f3-4037-8829-d12565545877 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:55.742209 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e1bf6b-a9f3-4037-8829-d12565545877 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:55.748001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-05e1bf6b-a9f3-4037-8829-d12565545877 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 05:51:55.748001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:55.748001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:55.748001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:55.748001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:55.748001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:55.748001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:55.748001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:55.748001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:55.748001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:55.748001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:55.748001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:55.748001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:55.748001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:55.748001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:55.748001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:55.749038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:55.749038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:55.749038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:55.749038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:55.749038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:55.749038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:55.749038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:55.749038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:55.749038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:55.749038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:55.749038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:55.749038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:55.749038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:55.749038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:55.749038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:55.749038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:55.749828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:55.749828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:55.749828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:55.749828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:55.749828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:55.749828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:55.749828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:55.749828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:55.749828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:55.749828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:55.749828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 05:51:55.749828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:51:55.749828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 140/272] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:51:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:51:55.756090 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1cd3a-1c0c-4fd5-a49b-da06bed749de admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:55.871813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:55.878778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1cd3a-1c0c-4fd5-a49b-da06bed749de admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:55.919779 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf2596-2ab6-42fb-8ce2-d57a2d2b9643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:55.920400 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf2596-2ab6-42fb-8ce2-d57a2d2b9643 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:55.920679 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf2596-2ab6-42fb-8ce2-d57a2d2b9643 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:55.920972 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf2596-2ab6-42fb-8ce2-d57a2d2b9643 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:55.921652 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bf2596-2ab6-42fb-8ce2-d57a2d2b9643 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:55.921947 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bf2596-2ab6-42fb-8ce2-d57a2d2b9643 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:55.923324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bf2596-2ab6-42fb-8ce2-d57a2d2b9643 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:55.927144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8bcbf-539e-44e5-9385-fc956fa382b2 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:55.927422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8bcbf-539e-44e5-9385-fc956fa382b2 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:55.927691 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8bcbf-539e-44e5-9385-fc956fa382b2 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:55.929097 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 141/273] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:55.937050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588eb31-3d65-4a57-8386-7ed0fb51f6c4 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:55.937262 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588eb31-3d65-4a57-8386-7ed0fb51f6c4 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:55.937533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588eb31-3d65-4a57-8386-7ed0fb51f6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:55.943744 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 133/274] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Oct 20 05:51:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:55.949261 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf2596-2ab6-42fb-8ce2-d57a2d2b9643 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:55.966275 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-5588eb31-3d65-4a57-8386-7ed0fb51f6c4 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:55.967472 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5588eb31-3d65-4a57-8386-7ed0fb51f6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:56.044310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 142/275] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 20 05:51:56.070076 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf2596-2ab6-42fb-8ce2-d57a2d2b9643 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:56.106926 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171f8b1-e6a2-4538-bb89-fcdf91d76458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:56.107809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171f8b1-e6a2-4538-bb89-fcdf91d76458 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:56.108258 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171f8b1-e6a2-4538-bb89-fcdf91d76458 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:56.108673 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171f8b1-e6a2-4538-bb89-fcdf91d76458 None admin] PATH_INFO: `/v3/users/9de92c67be1546b7b383527b7ebaf191` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:56.109985 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171f8b1-e6a2-4538-bb89-fcdf91d76458 None admin] RBAC: Authorizing `identity:get_user(user_id=9de92c67be1546b7b383527b7ebaf191)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:56.111595 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171f8b1-e6a2-4538-bb89-fcdf91d76458 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:56.114121 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 134/276] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:51:55 2022] GET /identity/v3/users/9de92c67be1546b7b383527b7ebaf191 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:56.120608 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588eb31-3d65-4a57-8386-7ed0fb51f6c4 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:56.235282 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:56.277697 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588eb31-3d65-4a57-8386-7ed0fb51f6c4 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:56.321381 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b381e-a918-4d61-9f5c-1c0f8fa5fa92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:56.321998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b381e-a918-4d61-9f5c-1c0f8fa5fa92 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:56.322277 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b381e-a918-4d61-9f5c-1c0f8fa5fa92 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:56.322546 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b381e-a918-4d61-9f5c-1c0f8fa5fa92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:56.323271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b381e-a918-4d61-9f5c-1c0f8fa5fa92 None admin] RBAC: Adding query filter params (role.id=78820dc7bd3841cfbc80e594e4e77171, user.id=9de92c67be1546b7b383527b7ebaf191, scope.system=all) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:56.323577 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b381e-a918-4d61-9f5c-1c0f8fa5fa92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:56.325048 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b381e-a918-4d61-9f5c-1c0f8fa5fa92 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:56.342011 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 143/277] 10.222.0.20 () {56 vars in 1308 bytes} [Thu Oct 20 05:51:56 2022] GET /identity/v3/role_assignments?user.id=9de92c67be1546b7b383527b7ebaf191&scope.system=all&role.id=78820dc7bd3841cfbc80e594e4e77171 => generated 510 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:56.397080 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171f8b1-e6a2-4538-bb89-fcdf91d76458 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:56.413527 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171f8b1-e6a2-4538-bb89-fcdf91d76458 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:56.418293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05708d3-580c-40d4-9351-7ee5add75e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:56.418958 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05708d3-580c-40d4-9351-7ee5add75e9c admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:56.419250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05708d3-580c-40d4-9351-7ee5add75e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:56.419525 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05708d3-580c-40d4-9351-7ee5add75e9c admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:56.434973 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05708d3-580c-40d4-9351-7ee5add75e9c admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:56.440046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-f05708d3-580c-40d4-9351-7ee5add75e9c admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:51:56.440046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:56.440046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:56.440046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:56.440046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:56.440046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:56.440046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:56.440046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:56.440046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:56.440046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:56.440046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:56.440046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:56.440046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:56.440046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:56.440046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:56.440046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:56.441586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:56.441586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:56.441586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:56.441586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:56.441586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:56.441586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:56.441586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:56.441586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:56.441586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:56.441586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:56.441586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:56.441586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:56.441586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:56.441586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:56.441586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:56.441586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:56.442368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:56.442368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:56.442368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:56.442368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:56.442368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:56.442368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:56.442368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:56.442368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:56.442368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:56.442368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:56.442368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:51:56.442368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:51:56.442368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 135/278] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Oct 20 05:51:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 46 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:51:56.448083 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b381e-a918-4d61-9f5c-1c0f8fa5fa92 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:56.465972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b381e-a918-4d61-9f5c-1c0f8fa5fa92 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:56.470961 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be9722-ae91-4c35-94ed-f39f22fbc63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:56.471608 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be9722-ae91-4c35-94ed-f39f22fbc63d admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:56.472411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be9722-ae91-4c35-94ed-f39f22fbc63d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:56.472707 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be9722-ae91-4c35-94ed-f39f22fbc63d admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:56.473430 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be9722-ae91-4c35-94ed-f39f22fbc63d admin admin] RBAC: Adding query filter params (name=member) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:56.473734 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be9722-ae91-4c35-94ed-f39f22fbc63d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:56.475441 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be9722-ae91-4c35-94ed-f39f22fbc63d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:56.489403 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 144/279] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Oct 20 05:51:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:56.494801 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05708d3-580c-40d4-9351-7ee5add75e9c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:56.509996 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05708d3-580c-40d4-9351-7ee5add75e9c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:56.514693 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465a10f-e5f1-4a22-8262-19c8ae01ad2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:56.515354 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465a10f-e5f1-4a22-8262-19c8ae01ad2d admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:56.515632 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465a10f-e5f1-4a22-8262-19c8ae01ad2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:56.515913 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465a10f-e5f1-4a22-8262-19c8ae01ad2d admin admin] PATH_INFO: `/v3/groups/8cdaa84a20c045fe900b1d8633f92fc4` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:56.517153 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465a10f-e5f1-4a22-8262-19c8ae01ad2d admin admin] RBAC: Authorizing `identity:get_group(group_id=8cdaa84a20c045fe900b1d8633f92fc4)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:56.519206 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465a10f-e5f1-4a22-8262-19c8ae01ad2d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:56.521168 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 136/280] 10.222.0.20 () {56 vars in 1201 bytes} [Thu Oct 20 05:51:56 2022] GET /identity/v3/groups/8cdaa84a20c045fe900b1d8633f92fc4 => generated 226 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:56.526827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be9722-ae91-4c35-94ed-f39f22fbc63d admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:56.537961 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be9722-ae91-4c35-94ed-f39f22fbc63d admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:56.546730 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e0ee3-628f-4676-8a5a-a1889e87203b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:56.547389 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e0ee3-628f-4676-8a5a-a1889e87203b admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:56.547668 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e0ee3-628f-4676-8a5a-a1889e87203b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:56.547942 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e0ee3-628f-4676-8a5a-a1889e87203b admin admin] PATH_INFO: `/v3/projects/e5ab74dfb5014b01bea8909bd1d64559` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:56.549135 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4e0ee3-628f-4676-8a5a-a1889e87203b admin admin] RBAC: Authorizing `identity:get_project(project_id=e5ab74dfb5014b01bea8909bd1d64559)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:56.551333 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4e0ee3-628f-4676-8a5a-a1889e87203b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:56.553224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 145/281] 10.222.0.20 () {56 vars in 1207 bytes} [Thu Oct 20 05:51:56 2022] GET /identity/v3/projects/e5ab74dfb5014b01bea8909bd1d64559 => generated 302 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:56.558760 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465a10f-e5f1-4a22-8262-19c8ae01ad2d admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:56.569522 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465a10f-e5f1-4a22-8262-19c8ae01ad2d admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:56.576504 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30190bb-4b18-4a87-b822-0cccdca355dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:56.577232 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30190bb-4b18-4a87-b822-0cccdca355dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:56.577512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30190bb-4b18-4a87-b822-0cccdca355dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:56.577787 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30190bb-4b18-4a87-b822-0cccdca355dd admin admin] PATH_INFO: `/v3/projects/e5ab74dfb5014b01bea8909bd1d64559/groups/8cdaa84a20c045fe900b1d8633f92fc4/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:56.579712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30190bb-4b18-4a87-b822-0cccdca355dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5ab74dfb5014b01bea8909bd1d64559, group_id=8cdaa84a20c045fe900b1d8633f92fc4, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:56.582812 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30190bb-4b18-4a87-b822-0cccdca355dd admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:56.589080 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:56.602050 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 137/282] 10.222.0.20 () {58 vars in 1463 bytes} [Thu Oct 20 05:51:56 2022] PUT /identity/v3/projects/e5ab74dfb5014b01bea8909bd1d64559/groups/8cdaa84a20c045fe900b1d8633f92fc4/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:51:56.615029 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68b869-c57e-4eb1-ba23-ce54f054d933 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:56.615413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68b869-c57e-4eb1-ba23-ce54f054d933 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:56.615762 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68b869-c57e-4eb1-ba23-ce54f054d933 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:56.617571 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 146/283] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:56.626726 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e87e7-e858-4f2e-a105-12f45afe3d29 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:56.626726 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e87e7-e858-4f2e-a105-12f45afe3d29 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:56.626726 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e87e7-e858-4f2e-a105-12f45afe3d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:56.650712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c25e87e7-e858-4f2e-a105-12f45afe3d29 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:56.652279 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c25e87e7-e858-4f2e-a105-12f45afe3d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:56.725950 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 138/284] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 20 05:51:57.028021 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68b869-c57e-4eb1-ba23-ce54f054d933 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:57.046344 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68b869-c57e-4eb1-ba23-ce54f054d933 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:57.051293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac99236-1a1f-4a78-96b3-2d46da9b9780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:57.051938 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac99236-1a1f-4a78-96b3-2d46da9b9780 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:57.052207 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac99236-1a1f-4a78-96b3-2d46da9b9780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:57.052851 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac99236-1a1f-4a78-96b3-2d46da9b9780 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:57.069550 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac99236-1a1f-4a78-96b3-2d46da9b9780 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:57.074439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-cac99236-1a1f-4a78-96b3-2d46da9b9780 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 05:51:57.074439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:57.074439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:57.074439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:57.074439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:57.074439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:57.074439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:57.074439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:57.074439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:57.074439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:57.074439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:57.074439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:57.074439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:57.074439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:57.074439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:57.074439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:57.075289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:57.075289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:57.075289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:57.075289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:57.075289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:57.075289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:57.075289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:57.075289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:57.075289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:57.075289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:57.075289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:57.075289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:57.075289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:57.075289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:57.075289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:57.075289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:57.076090 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:57.076090 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:57.076090 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:57.076090 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:57.076090 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:57.076090 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:57.076090 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:57.076090 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:57.076090 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:57.076090 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:57.076090 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 05:51:57.076090 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:51:57.078150 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 147/285] 10.222.0.20 () {56 vars in 1135 bytes} [Thu Oct 20 05:51:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 48 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:51:57.083939 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e87e7-e858-4f2e-a105-12f45afe3d29 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:57.098947 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e87e7-e858-4f2e-a105-12f45afe3d29 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:57.103976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68eef9-0fb8-46e2-b1b0-03002bb81520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:57.104859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68eef9-0fb8-46e2-b1b0-03002bb81520 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:57.105333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68eef9-0fb8-46e2-b1b0-03002bb81520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:57.105814 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68eef9-0fb8-46e2-b1b0-03002bb81520 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:57.106724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba68eef9-0fb8-46e2-b1b0-03002bb81520 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:57.107224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba68eef9-0fb8-46e2-b1b0-03002bb81520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:57.113600 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba68eef9-0fb8-46e2-b1b0-03002bb81520 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:57.124203 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 139/286] 10.222.0.20 () {56 vars in 1132 bytes} [Thu Oct 20 05:51:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:57.131351 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac99236-1a1f-4a78-96b3-2d46da9b9780 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:57.147328 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac99236-1a1f-4a78-96b3-2d46da9b9780 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:57.156727 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976166cd-f552-477f-a805-670cec65ada2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:57.157396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976166cd-f552-477f-a805-670cec65ada2 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:57.157674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976166cd-f552-477f-a805-670cec65ada2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:57.157947 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976166cd-f552-477f-a805-670cec65ada2 admin admin] PATH_INFO: `/v3/groups/8cdaa84a20c045fe900b1d8633f92fc4` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:57.160888 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976166cd-f552-477f-a805-670cec65ada2 admin admin] RBAC: Authorizing `identity:get_group(group_id=8cdaa84a20c045fe900b1d8633f92fc4)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:57.162813 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976166cd-f552-477f-a805-670cec65ada2 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:57.165461 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 148/287] 10.222.0.20 () {56 vars in 1201 bytes} [Thu Oct 20 05:51:57 2022] GET /identity/v3/groups/8cdaa84a20c045fe900b1d8633f92fc4 => generated 226 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:57.170277 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68eef9-0fb8-46e2-b1b0-03002bb81520 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:57.185351 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68eef9-0fb8-46e2-b1b0-03002bb81520 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:57.190319 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ebb9e-7c8c-4602-9344-d43122cf64cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:57.191770 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ebb9e-7c8c-4602-9344-d43122cf64cc admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:57.192244 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ebb9e-7c8c-4602-9344-d43122cf64cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:57.192757 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ebb9e-7c8c-4602-9344-d43122cf64cc admin admin] PATH_INFO: `/v3/projects/e5ab74dfb5014b01bea8909bd1d64559` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:57.194191 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ebb9e-7c8c-4602-9344-d43122cf64cc admin admin] RBAC: Authorizing `identity:get_project(project_id=e5ab74dfb5014b01bea8909bd1d64559)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:57.201486 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ebb9e-7c8c-4602-9344-d43122cf64cc admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:57.203524 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 140/288] 10.222.0.20 () {56 vars in 1207 bytes} [Thu Oct 20 05:51:57 2022] GET /identity/v3/projects/e5ab74dfb5014b01bea8909bd1d64559 => generated 302 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:57.210302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976166cd-f552-477f-a805-670cec65ada2 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:57.222957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976166cd-f552-477f-a805-670cec65ada2 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:57.232835 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41369eeb-7efc-4241-a427-b9dac8054ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:57.233513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41369eeb-7efc-4241-a427-b9dac8054ce9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:57.233793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41369eeb-7efc-4241-a427-b9dac8054ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:57.234069 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41369eeb-7efc-4241-a427-b9dac8054ce9 admin admin] PATH_INFO: `/v3/projects/e5ab74dfb5014b01bea8909bd1d64559/groups/8cdaa84a20c045fe900b1d8633f92fc4/roles/49dae0405c764c96b7bfbcf9d9efd011` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:57.242519 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41369eeb-7efc-4241-a427-b9dac8054ce9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5ab74dfb5014b01bea8909bd1d64559, group_id=8cdaa84a20c045fe900b1d8633f92fc4, role_id=49dae0405c764c96b7bfbcf9d9efd011)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:57.245352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41369eeb-7efc-4241-a427-b9dac8054ce9 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:57.267678 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 149/289] 10.222.0.20 () {58 vars in 1463 bytes} [Thu Oct 20 05:51:57 2022] PUT /identity/v3/projects/e5ab74dfb5014b01bea8909bd1d64559/groups/8cdaa84a20c045fe900b1d8633f92fc4/roles/49dae0405c764c96b7bfbcf9d9efd011 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:51:57.406420 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee081f0-83c2-4a7c-b3c1-4b5d4b2ae6bf None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:57.406739 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee081f0-83c2-4a7c-b3c1-4b5d4b2ae6bf None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:57.406967 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee081f0-83c2-4a7c-b3c1-4b5d4b2ae6bf None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:57.408447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 141/290] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:57.413949 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb0107-9e4d-442b-a565-f204583c85fd None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:57.414235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb0107-9e4d-442b-a565-f204583c85fd None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:57.414810 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb0107-9e4d-442b-a565-f204583c85fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:57.443654 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-43eb0107-9e4d-442b-a565-f204583c85fd None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:57.445013 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-43eb0107-9e4d-442b-a565-f204583c85fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:57.523078 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 150/291] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 20 05:51:57.918176 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee081f0-83c2-4a7c-b3c1-4b5d4b2ae6bf None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:57.931001 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee081f0-83c2-4a7c-b3c1-4b5d4b2ae6bf None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:57.935883 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0d14d-6024-4f29-8699-a33fc3c5d1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:57.941063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0d14d-6024-4f29-8699-a33fc3c5d1fb admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:57.941743 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0d14d-6024-4f29-8699-a33fc3c5d1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:57.942277 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0d14d-6024-4f29-8699-a33fc3c5d1fb admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:57.958175 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0d14d-6024-4f29-8699-a33fc3c5d1fb admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:57.964968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-c1b0d14d-6024-4f29-8699-a33fc3c5d1fb admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:51:57.964968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:57.964968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:57.964968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:57.964968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:57.964968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:57.964968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:57.964968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:57.964968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:57.964968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:57.964968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:57.964968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:57.964968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:57.964968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:57.964968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:57.964968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:57.965827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:57.965827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:57.965827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:57.965827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:57.965827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:57.965827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:57.965827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:57.965827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:57.965827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:57.965827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:57.965827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:57.965827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:57.965827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:57.965827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:57.965827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:57.965827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:57.966613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:57.966613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:57.966613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:57.966613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:57.966613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:57.966613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:57.966613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:57.966613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:57.966613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:57.966613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:57.966613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:51:57.966613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:51:57.972662 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 142/292] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Oct 20 05:51:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 55 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:51:57.978696 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eb0107-9e4d-442b-a565-f204583c85fd None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:57.988719 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eb0107-9e4d-442b-a565-f204583c85fd None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:58.001817 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436be5dc-a8e8-4aba-aea2-d47b833ef381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:58.002438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436be5dc-a8e8-4aba-aea2-d47b833ef381 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:58.002710 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436be5dc-a8e8-4aba-aea2-d47b833ef381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:58.003535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436be5dc-a8e8-4aba-aea2-d47b833ef381 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:58.004182 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436be5dc-a8e8-4aba-aea2-d47b833ef381 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:58.004514 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436be5dc-a8e8-4aba-aea2-d47b833ef381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:58.006200 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436be5dc-a8e8-4aba-aea2-d47b833ef381 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:58.026136 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 151/293] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Oct 20 05:51:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:58.031713 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0d14d-6024-4f29-8699-a33fc3c5d1fb admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:58.043141 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0d14d-6024-4f29-8699-a33fc3c5d1fb admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:58.047936 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e4ec8-f1e2-477f-85b1-a903d9d871b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:58.048825 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e4ec8-f1e2-477f-85b1-a903d9d871b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:58.049301 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e4ec8-f1e2-477f-85b1-a903d9d871b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:58.049767 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e4ec8-f1e2-477f-85b1-a903d9d871b1 admin admin] PATH_INFO: `/v3/groups/2e0ca34b71b34feaa4cc7634226e7232` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:58.057184 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829e4ec8-f1e2-477f-85b1-a903d9d871b1 admin admin] RBAC: Authorizing `identity:get_group(group_id=2e0ca34b71b34feaa4cc7634226e7232)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:58.059352 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829e4ec8-f1e2-477f-85b1-a903d9d871b1 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:58.061513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 143/294] 10.222.0.20 () {56 vars in 1201 bytes} [Thu Oct 20 05:51:58 2022] GET /identity/v3/groups/2e0ca34b71b34feaa4cc7634226e7232 => generated 229 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:58.067157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436be5dc-a8e8-4aba-aea2-d47b833ef381 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:58.077470 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436be5dc-a8e8-4aba-aea2-d47b833ef381 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:58.089038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9920bc9-3f6f-48f6-8c7e-ceecd58f21ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:58.089691 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9920bc9-3f6f-48f6-8c7e-ceecd58f21ea admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:58.089970 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9920bc9-3f6f-48f6-8c7e-ceecd58f21ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:58.090244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9920bc9-3f6f-48f6-8c7e-ceecd58f21ea admin admin] PATH_INFO: `/v3/projects/edda8534137f405d818dbc74b7391c73` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:58.097595 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9920bc9-3f6f-48f6-8c7e-ceecd58f21ea admin admin] RBAC: Authorizing `identity:get_project(project_id=edda8534137f405d818dbc74b7391c73)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:58.099407 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9920bc9-3f6f-48f6-8c7e-ceecd58f21ea admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:58.101456 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 152/295] 10.222.0.20 () {56 vars in 1207 bytes} [Thu Oct 20 05:51:58 2022] GET /identity/v3/projects/edda8534137f405d818dbc74b7391c73 => generated 344 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:58.106894 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e4ec8-f1e2-477f-85b1-a903d9d871b1 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:58.122067 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e4ec8-f1e2-477f-85b1-a903d9d871b1 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:58.130760 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2548487-f1b9-411b-92d4-3ea844fb4edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:58.131566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2548487-f1b9-411b-92d4-3ea844fb4edd admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:58.131848 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2548487-f1b9-411b-92d4-3ea844fb4edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:58.132117 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2548487-f1b9-411b-92d4-3ea844fb4edd admin admin] PATH_INFO: `/v3/projects/edda8534137f405d818dbc74b7391c73/groups/2e0ca34b71b34feaa4cc7634226e7232/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:58.134911 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2548487-f1b9-411b-92d4-3ea844fb4edd admin admin] RBAC: Authorizing `identity:create_grant(project_id=edda8534137f405d818dbc74b7391c73, group_id=2e0ca34b71b34feaa4cc7634226e7232, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:58.137625 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2548487-f1b9-411b-92d4-3ea844fb4edd admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:58.157495 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 144/296] 10.222.0.20 () {58 vars in 1463 bytes} [Thu Oct 20 05:51:58 2022] PUT /identity/v3/projects/edda8534137f405d818dbc74b7391c73/groups/2e0ca34b71b34feaa4cc7634226e7232/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:51:58.620824 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b88e2-52cd-4b65-9b53-5207fc513400 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:58.621116 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b88e2-52cd-4b65-9b53-5207fc513400 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:58.621463 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b88e2-52cd-4b65-9b53-5207fc513400 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:58.622813 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 153/297] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:58.628680 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89313ea8-0e72-4690-8e08-a29e6cbf38f5 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:58.629016 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89313ea8-0e72-4690-8e08-a29e6cbf38f5 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:58.629293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89313ea8-0e72-4690-8e08-a29e6cbf38f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:58.650270 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-89313ea8-0e72-4690-8e08-a29e6cbf38f5 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:58.651464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-89313ea8-0e72-4690-8e08-a29e6cbf38f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:58.755278 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 145/298] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:58.849241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83809ddb-c36c-4441-b1bb-f338cb2fd8dc None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:58.849561 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83809ddb-c36c-4441-b1bb-f338cb2fd8dc None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:58.849790 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83809ddb-c36c-4441-b1bb-f338cb2fd8dc None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:58.851347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 154/299] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:58.857270 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115da69-de62-4c48-b437-575e11b57bc1 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:58.857554 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115da69-de62-4c48-b437-575e11b57bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:58.857828 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115da69-de62-4c48-b437-575e11b57bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:58.885796 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c115da69-de62-4c48-b437-575e11b57bc1 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:58.886993 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c115da69-de62-4c48-b437-575e11b57bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:58.915622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:58.980147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 146/300] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:59.204431 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83809ddb-c36c-4441-b1bb-f338cb2fd8dc None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:59.225881 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:59.353484 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83809ddb-c36c-4441-b1bb-f338cb2fd8dc None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:59.400643 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e694d80-9b36-43c2-b9c0-49d4a641223d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:59.401304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e694d80-9b36-43c2-b9c0-49d4a641223d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:59.401579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e694d80-9b36-43c2-b9c0-49d4a641223d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:59.401845 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e694d80-9b36-43c2-b9c0-49d4a641223d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:59.418080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e694d80-9b36-43c2-b9c0-49d4a641223d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:59.423268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-7e694d80-9b36-43c2-b9c0-49d4a641223d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:51:59.423268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:59.423268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:59.423268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:59.423268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:59.423268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:59.423268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:59.423268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:59.423268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:59.423268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:59.423268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:59.423268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:59.423268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:59.423268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:59.423268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:59.423268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:59.424126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:59.424126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:59.424126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:59.424126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:59.424126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:59.424126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:59.424126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:59.424126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:59.424126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:59.424126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:59.424126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:59.424126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:59.424126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:59.424126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:59.424126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:59.424126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:59.425122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:59.425122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:59.425122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:59.425122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:59.425122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:59.425122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:59.425122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:59.425122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:59.425122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:59.425122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:59.425122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:51:59.425122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:51:59.427570 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 155/301] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:51:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:51:59.433567 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115da69-de62-4c48-b437-575e11b57bc1 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:59.439377 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e694d80-9b36-43c2-b9c0-49d4a641223d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:59.564768 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:59.599020 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115da69-de62-4c48-b437-575e11b57bc1 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:59.600834 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e694d80-9b36-43c2-b9c0-49d4a641223d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:59.637402 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be3ed9-57f6-4182-ab37-955ae0656df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:59.638021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be3ed9-57f6-4182-ab37-955ae0656df1 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:59.638303 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be3ed9-57f6-4182-ab37-955ae0656df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:59.638629 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be3ed9-57f6-4182-ab37-955ae0656df1 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:59.639239 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be3ed9-57f6-4182-ab37-955ae0656df1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:59.639545 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be3ed9-57f6-4182-ab37-955ae0656df1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:59.641560 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be3ed9-57f6-4182-ab37-955ae0656df1 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:59.648135 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35836f4d-bc56-4536-9e80-e9993c55854e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:59.649312 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35836f4d-bc56-4536-9e80-e9993c55854e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:59.649791 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35836f4d-bc56-4536-9e80-e9993c55854e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:59.650252 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35836f4d-bc56-4536-9e80-e9993c55854e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:59.650976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 147/302] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Oct 20 05:51:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:59.652373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809f395-70ef-45b8-8e4d-3be904969b60 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:59.652790 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809f395-70ef-45b8-8e4d-3be904969b60 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:59.652961 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809f395-70ef-45b8-8e4d-3be904969b60 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:59.655096 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 148/303] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:51:59.660242 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809f395-70ef-45b8-8e4d-3be904969b60 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:59.667966 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35836f4d-bc56-4536-9e80-e9993c55854e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:51:59.675863 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-35836f4d-bc56-4536-9e80-e9993c55854e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 05:51:59.675863 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:51:59.675863 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:51:59.675863 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:51:59.675863 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:51:59.675863 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:51:59.675863 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:51:59.675863 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:51:59.675863 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:51:59.675863 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:51:59.675863 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:51:59.675863 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:51:59.675863 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:51:59.675863 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:51:59.675863 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:51:59.675863 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:51:59.676770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:51:59.676770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:51:59.676770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:51:59.676770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:51:59.676770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:51:59.676770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:51:59.676770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:51:59.676770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:51:59.676770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:51:59.676770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:51:59.676770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:51:59.676770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:51:59.676770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:51:59.676770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:51:59.676770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:51:59.676770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:51:59.677563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:51:59.677563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:51:59.677563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:51:59.677563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:51:59.677563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:51:59.677563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:51:59.677563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:51:59.677563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:51:59.677563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:51:59.677563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:51:59.677563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 05:51:59.677563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:51:59.679971 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 156/304] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Oct 20 05:51:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:51:59.681766 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b599f39-1457-4c69-a334-102af2122f4e None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:59.682261 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b599f39-1457-4c69-a334-102af2122f4e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:59.682735 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b599f39-1457-4c69-a334-102af2122f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:59.688269 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:59.706586 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-0b599f39-1457-4c69-a334-102af2122f4e None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:51:59.708570 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-0b599f39-1457-4c69-a334-102af2122f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:51:59.793376 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809f395-70ef-45b8-8e4d-3be904969b60 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:59.795993 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 157/305] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:51:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:51:59.797213 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b599f39-1457-4c69-a334-102af2122f4e None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:51:59.837288 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9309e2a-82f9-4e05-a7e9-b494d9ab967c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:59.838375 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9309e2a-82f9-4e05-a7e9-b494d9ab967c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:59.838848 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9309e2a-82f9-4e05-a7e9-b494d9ab967c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:59.839327 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9309e2a-82f9-4e05-a7e9-b494d9ab967c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:59.877581 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:59.917442 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b599f39-1457-4c69-a334-102af2122f4e None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:51:59.933160 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9309e2a-82f9-4e05-a7e9-b494d9ab967c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:59.935597 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9309e2a-82f9-4e05-a7e9-b494d9ab967c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:59.957516 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:51:59.973033 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77ba57-8b71-4183-95f9-c3828e96f371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:51:59.973981 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77ba57-8b71-4183-95f9-c3828e96f371 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:51:59.974467 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77ba57-8b71-4183-95f9-c3828e96f371 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:51:59.974939 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77ba57-8b71-4183-95f9-c3828e96f371 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:51:59.975843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77ba57-8b71-4183-95f9-c3828e96f371 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:51:59.976360 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77ba57-8b71-4183-95f9-c3828e96f371 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:51:59.978033 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77ba57-8b71-4183-95f9-c3828e96f371 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:51:59.987318 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 158/306] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:51:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:51:59.993656 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77ba57-8b71-4183-95f9-c3828e96f371 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:00.114385 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 149/307] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:51:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 05:52:00.119925 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9309e2a-82f9-4e05-a7e9-b494d9ab967c None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:00.192336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77ba57-8b71-4183-95f9-c3828e96f371 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:00.230605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:00.241005 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc87331-9fdb-494a-9f90-4c0c527e9ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:00.242724 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc87331-9fdb-494a-9f90-4c0c527e9ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:00.243284 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc87331-9fdb-494a-9f90-4c0c527e9ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:00.244092 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc87331-9fdb-494a-9f90-4c0c527e9ef5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:00.273961 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9309e2a-82f9-4e05-a7e9-b494d9ab967c None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:00.307453 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:00.342434 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661903a-1e22-4b9e-9795-e0702c5d49d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:00.344092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661903a-1e22-4b9e-9795-e0702c5d49d9 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:00.345005 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661903a-1e22-4b9e-9795-e0702c5d49d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:00.345449 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661903a-1e22-4b9e-9795-e0702c5d49d9 None admin] PATH_INFO: `/v3/projects/e78108de05fe409aa6e56461abb56cc0` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:00.347524 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661903a-1e22-4b9e-9795-e0702c5d49d9 None admin] RBAC: Authorizing `identity:get_project(project_id=e78108de05fe409aa6e56461abb56cc0)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:00.350211 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661903a-1e22-4b9e-9795-e0702c5d49d9 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:00.352454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 150/308] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:52:00 2022] GET /identity/v3/projects/e78108de05fe409aa6e56461abb56cc0 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:00.353740 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc87331-9fdb-494a-9f90-4c0c527e9ef5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:00.353863 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661903a-1e22-4b9e-9795-e0702c5d49d9 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:00.355215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc87331-9fdb-494a-9f90-4c0c527e9ef5 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:00.476638 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 159/309] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:51:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 20 05:52:00.478553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc87331-9fdb-494a-9f90-4c0c527e9ef5 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:00.487919 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661903a-1e22-4b9e-9795-e0702c5d49d9 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:00.522997 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:00.538712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f3844-73f1-4a1f-9dea-d893dc24c219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:00.539819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f3844-73f1-4a1f-9dea-d893dc24c219 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:00.540149 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f3844-73f1-4a1f-9dea-d893dc24c219 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:00.540428 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f3844-73f1-4a1f-9dea-d893dc24c219 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:00.554148 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f3844-73f1-4a1f-9dea-d893dc24c219 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:52:00.559083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-f01f3844-73f1-4a1f-9dea-d893dc24c219 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:52:00.559083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:00.559083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:00.559083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:00.559083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:00.559083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:00.559083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:00.559083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:00.559083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:00.559083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:00.559083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:00.559083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:00.559083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:52:00.559083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:52:00.559083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:52:00.559083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:52:00.559971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:52:00.559971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:52:00.559971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:52:00.559971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:52:00.559971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:00.559971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:00.559971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:00.559971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:00.559971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:00.559971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:00.559971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:00.559971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:00.559971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:00.559971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:00.559971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:00.559971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:00.561089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:00.561089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:00.561089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:00.561089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:00.561089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:52:00.561089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:52:00.561089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:52:00.561089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:52:00.561089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:52:00.561089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:52:00.561089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 05:52:00.561089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:00.561089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 151/310] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:52:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:00.562263 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de703a-ea42-46bf-a2af-7687f8f82664 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:00.562263 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de703a-ea42-46bf-a2af-7687f8f82664 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:00.562429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de703a-ea42-46bf-a2af-7687f8f82664 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:00.563769 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 152/311] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:00.564311 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de703a-ea42-46bf-a2af-7687f8f82664 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:00.610430 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc87331-9fdb-494a-9f90-4c0c527e9ef5 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:00.633104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:00.654048 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e6f15e-b10c-4adc-b92e-5f073a544617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:00.654748 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e6f15e-b10c-4adc-b92e-5f073a544617 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:00.655027 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e6f15e-b10c-4adc-b92e-5f073a544617 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:00.655312 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e6f15e-b10c-4adc-b92e-5f073a544617 None admin] PATH_INFO: `/v3/groups/8cdaa84a20c045fe900b1d8633f92fc4` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:00.656453 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e6f15e-b10c-4adc-b92e-5f073a544617 None admin] RBAC: Authorizing `identity:get_group(group_id=8cdaa84a20c045fe900b1d8633f92fc4)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:00.658008 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e6f15e-b10c-4adc-b92e-5f073a544617 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:00.660023 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 160/312] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Oct 20 05:52:00 2022] GET /identity/v3/groups/8cdaa84a20c045fe900b1d8633f92fc4 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:00.660919 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e6f15e-b10c-4adc-b92e-5f073a544617 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:00.697742 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de703a-ea42-46bf-a2af-7687f8f82664 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:00.745484 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c2e97-c415-4fb9-9fad-044a3f69d2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:00.746136 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c2e97-c415-4fb9-9fad-044a3f69d2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:00.746416 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c2e97-c415-4fb9-9fad-044a3f69d2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:00.746685 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c2e97-c415-4fb9-9fad-044a3f69d2c1 None admin] PATH_INFO: `/v3/projects/e78108de05fe409aa6e56461abb56cc0` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:00.747841 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c2e97-c415-4fb9-9fad-044a3f69d2c1 None admin] RBAC: Authorizing `identity:get_project(project_id=e78108de05fe409aa6e56461abb56cc0)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:00.749401 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c2e97-c415-4fb9-9fad-044a3f69d2c1 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:00.751191 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 153/313] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:52:00 2022] GET /identity/v3/projects/e78108de05fe409aa6e56461abb56cc0 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:00.752163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c12c7a-a2ef-433a-b2ae-337278bebbc5 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:00.752440 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c12c7a-a2ef-433a-b2ae-337278bebbc5 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:00.752771 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c12c7a-a2ef-433a-b2ae-337278bebbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:00.772963 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f0c12c7a-a2ef-433a-b2ae-337278bebbc5 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:00.774161 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f0c12c7a-a2ef-433a-b2ae-337278bebbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:00.779315 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e6f15e-b10c-4adc-b92e-5f073a544617 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:00.781054 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:00.821004 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653a8a1-f40e-4130-9110-973a8ba3d2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:00.821990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653a8a1-f40e-4130-9110-973a8ba3d2c6 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:00.823069 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653a8a1-f40e-4130-9110-973a8ba3d2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:00.823447 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653a8a1-f40e-4130-9110-973a8ba3d2c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:00.824323 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653a8a1-f40e-4130-9110-973a8ba3d2c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:00.824764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653a8a1-f40e-4130-9110-973a8ba3d2c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:00.826300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653a8a1-f40e-4130-9110-973a8ba3d2c6 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:00.836282 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 161/314] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Oct 20 05:52:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:00.837374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653a8a1-f40e-4130-9110-973a8ba3d2c6 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:00.863005 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 154/315] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:52:00.864198 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c12c7a-a2ef-433a-b2ae-337278bebbc5 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:00.926146 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:00.967809 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653a8a1-f40e-4130-9110-973a8ba3d2c6 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:01.000740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c12c7a-a2ef-433a-b2ae-337278bebbc5 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:01.008427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6c9b8-b44e-467f-a3cd-d2afa354cd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:01.009066 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6c9b8-b44e-467f-a3cd-d2afa354cd06 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:01.009344 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6c9b8-b44e-467f-a3cd-d2afa354cd06 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:01.009859 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6c9b8-b44e-467f-a3cd-d2afa354cd06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:01.010584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6c9b8-b44e-467f-a3cd-d2afa354cd06 None admin] RBAC: Adding query filter params (group.id=8cdaa84a20c045fe900b1d8633f92fc4, role.id=c1e0df53e1264ec79d91065e25acd29d, scope.project.id=e78108de05fe409aa6e56461abb56cc0) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:01.010888 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6c9b8-b44e-467f-a3cd-d2afa354cd06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:01.012437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6c9b8-b44e-467f-a3cd-d2afa354cd06 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:01.020025 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 162/316] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Oct 20 05:52:00 2022] GET /identity/v3/role_assignments?group.id=8cdaa84a20c045fe900b1d8633f92fc4&scope.project.id=e78108de05fe409aa6e56461abb56cc0&role.id=c1e0df53e1264ec79d91065e25acd29d => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:01.021380 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6c9b8-b44e-467f-a3cd-d2afa354cd06 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:01.051297 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c27ff3-7e67-4613-a028-be127efd7622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:01.052415 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c27ff3-7e67-4613-a028-be127efd7622 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:01.052824 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:01.053026 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c27ff3-7e67-4613-a028-be127efd7622 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:01.053454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c27ff3-7e67-4613-a028-be127efd7622 None admin] PATH_INFO: `/v3/groups/8cdaa84a20c045fe900b1d8633f92fc4` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:01.055657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c27ff3-7e67-4613-a028-be127efd7622 None admin] RBAC: Authorizing `identity:get_group(group_id=8cdaa84a20c045fe900b1d8633f92fc4)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:01.058012 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c27ff3-7e67-4613-a028-be127efd7622 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:01.060367 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 155/317] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Oct 20 05:52:00 2022] GET /identity/v3/groups/8cdaa84a20c045fe900b1d8633f92fc4 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:01.070018 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c27ff3-7e67-4613-a028-be127efd7622 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:01.150735 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6c9b8-b44e-467f-a3cd-d2afa354cd06 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:01.202767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26783699-40aa-4620-a1f7-76fb622e272c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:01.203396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26783699-40aa-4620-a1f7-76fb622e272c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:01.203672 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26783699-40aa-4620-a1f7-76fb622e272c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:01.203942 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26783699-40aa-4620-a1f7-76fb622e272c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:01.236331 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c27ff3-7e67-4613-a028-be127efd7622 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:01.253416 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:01.282226 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26783699-40aa-4620-a1f7-76fb622e272c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:01.283696 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26783699-40aa-4620-a1f7-76fb622e272c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:01.286429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb9d8c-bec9-471b-b389-c191a5f289b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:01.288195 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb9d8c-bec9-471b-b389-c191a5f289b5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:01.288675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb9d8c-bec9-471b-b389-c191a5f289b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:01.289147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb9d8c-bec9-471b-b389-c191a5f289b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:01.290077 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedb9d8c-bec9-471b-b389-c191a5f289b5 None admin] RBAC: Adding query filter params (group.id=8cdaa84a20c045fe900b1d8633f92fc4, role.id=49dae0405c764c96b7bfbcf9d9efd011, scope.project.id=e78108de05fe409aa6e56461abb56cc0) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:01.290619 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedb9d8c-bec9-471b-b389-c191a5f289b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:01.292235 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedb9d8c-bec9-471b-b389-c191a5f289b5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:01.302715 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 156/318] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Oct 20 05:52:01 2022] GET /identity/v3/role_assignments?group.id=8cdaa84a20c045fe900b1d8633f92fc4&scope.project.id=e78108de05fe409aa6e56461abb56cc0&role.id=49dae0405c764c96b7bfbcf9d9efd011 => generated 611 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:01.304350 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c7437-2e69-46ab-af98-679d129acaef None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:01.304890 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c7437-2e69-46ab-af98-679d129acaef None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:01.305378 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c7437-2e69-46ab-af98-679d129acaef None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:01.310032 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 157/319] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:01.310032 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c7437-2e69-46ab-af98-679d129acaef None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:01.319950 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:01.440111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 163/320] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:52:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 05:52:01.441663 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3807ae-deee-4b12-941a-9fd72adea5f8 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:01.441944 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3807ae-deee-4b12-941a-9fd72adea5f8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:01.442214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3807ae-deee-4b12-941a-9fd72adea5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:01.455583 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c7437-2e69-46ab-af98-679d129acaef None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:01.465449 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-2e3807ae-deee-4b12-941a-9fd72adea5f8 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:01.466686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2e3807ae-deee-4b12-941a-9fd72adea5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:01.501084 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c8312-5bfd-4d6f-8a3c-b08cda318ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:01.501736 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c8312-5bfd-4d6f-8a3c-b08cda318ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:01.502037 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c8312-5bfd-4d6f-8a3c-b08cda318ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:01.502288 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c8312-5bfd-4d6f-8a3c-b08cda318ac6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:01.513809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442c8312-5bfd-4d6f-8a3c-b08cda318ac6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:52:01.519202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-442c8312-5bfd-4d6f-8a3c-b08cda318ac6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 05:52:01.519202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:01.519202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:01.519202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:01.519202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:01.519202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:01.519202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:01.519202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:01.519202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:01.519202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:01.519202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:01.519202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:01.519202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:52:01.519202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:52:01.519202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:52:01.519202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:52:01.520209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:52:01.520209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:52:01.520209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:52:01.520209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:52:01.520209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:01.520209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:01.520209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:01.520209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:01.520209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:01.520209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:01.520209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:01.520209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:01.520209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:01.520209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:01.520209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:01.520209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:01.521058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:01.521058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:01.521058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:01.521058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:01.521058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:52:01.521058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:52:01.521058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:52:01.521058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:52:01.521058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:52:01.521058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:52:01.521058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 05:52:01.521058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:01.521664 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 158/321] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Oct 20 05:52:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:01.522220 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c8312-5bfd-4d6f-8a3c-b08cda318ac6 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:01.555875 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 164/322] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:52:01.557067 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3807ae-deee-4b12-941a-9fd72adea5f8 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:01.573460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:01.633869 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:01.654497 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c8312-5bfd-4d6f-8a3c-b08cda318ac6 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:01.687336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3807ae-deee-4b12-941a-9fd72adea5f8 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:01.708384 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d9356-a4c3-4dc6-8884-16dd0611f567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:01.709068 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d9356-a4c3-4dc6-8884-16dd0611f567 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:01.709340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d9356-a4c3-4dc6-8884-16dd0611f567 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:01.709607 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d9356-a4c3-4dc6-8884-16dd0611f567 None admin] PATH_INFO: `/v3/projects/e5ab74dfb5014b01bea8909bd1d64559` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:01.710729 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d9356-a4c3-4dc6-8884-16dd0611f567 None admin] RBAC: Authorizing `identity:get_project(project_id=e5ab74dfb5014b01bea8909bd1d64559)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:01.712162 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d9356-a4c3-4dc6-8884-16dd0611f567 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:01.714132 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 159/323] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:52:01 2022] GET /identity/v3/projects/e5ab74dfb5014b01bea8909bd1d64559 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:01.720095 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d9356-a4c3-4dc6-8884-16dd0611f567 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:01.734878 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e005bf-9101-41e6-bc51-76cca076b2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:01.736949 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e005bf-9101-41e6-bc51-76cca076b2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:01.736949 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e005bf-9101-41e6-bc51-76cca076b2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:01.736949 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e005bf-9101-41e6-bc51-76cca076b2b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:01.737782 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e005bf-9101-41e6-bc51-76cca076b2b1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:01.738101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e005bf-9101-41e6-bc51-76cca076b2b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:01.740143 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e005bf-9101-41e6-bc51-76cca076b2b1 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:01.750714 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 165/324] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:01.758441 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e005bf-9101-41e6-bc51-76cca076b2b1 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:01.849192 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d9356-a4c3-4dc6-8884-16dd0611f567 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:01.852136 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:01.876561 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e005bf-9101-41e6-bc51-76cca076b2b1 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:01.890244 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab859c-d128-4fc7-9839-ec04be0186fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:01.891050 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab859c-d128-4fc7-9839-ec04be0186fe None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:01.891333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab859c-d128-4fc7-9839-ec04be0186fe None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:01.891609 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab859c-d128-4fc7-9839-ec04be0186fe None admin] PATH_INFO: `/v3/groups/8cdaa84a20c045fe900b1d8633f92fc4` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:01.892903 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddab859c-d128-4fc7-9839-ec04be0186fe None admin] RBAC: Authorizing `identity:get_group(group_id=8cdaa84a20c045fe900b1d8633f92fc4)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:01.895088 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddab859c-d128-4fc7-9839-ec04be0186fe None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:01.899189 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:01.900983 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 160/325] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Oct 20 05:52:01 2022] GET /identity/v3/groups/8cdaa84a20c045fe900b1d8633f92fc4 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:01.901962 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab859c-d128-4fc7-9839-ec04be0186fe None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:01.923979 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfb69e-0124-4af7-8de7-0db3544b7ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:01.924590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfb69e-0124-4af7-8de7-0db3544b7ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:01.925445 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfb69e-0124-4af7-8de7-0db3544b7ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:01.925723 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfb69e-0124-4af7-8de7-0db3544b7ee6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:02.005171 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcfb69e-0124-4af7-8de7-0db3544b7ee6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:02.006764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcfb69e-0124-4af7-8de7-0db3544b7ee6 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:02.021885 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab859c-d128-4fc7-9839-ec04be0186fe None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:02.060616 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366df0d6-0923-4ee5-9936-5bbd1144fc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:02.061292 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366df0d6-0923-4ee5-9936-5bbd1144fc09 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:02.063552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366df0d6-0923-4ee5-9936-5bbd1144fc09 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:02.063552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366df0d6-0923-4ee5-9936-5bbd1144fc09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:02.078990 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366df0d6-0923-4ee5-9936-5bbd1144fc09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:52:02.085831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-366df0d6-0923-4ee5-9936-5bbd1144fc09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:52:02.085831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:02.085831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:02.085831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:02.085831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:02.085831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:02.085831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:02.085831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:02.085831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:02.085831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:02.085831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:02.085831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:02.085831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:52:02.085831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:52:02.085831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:52:02.085831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:52:02.086701 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:52:02.086701 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:52:02.086701 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:52:02.086701 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:52:02.086701 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:02.086701 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:02.086701 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:02.086701 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:02.086701 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:02.086701 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:02.086701 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:02.086701 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:02.086701 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:02.086701 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:02.086701 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:02.086701 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:02.087493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:02.087493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:02.087493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:02.087493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:02.087493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:52:02.087493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:52:02.087493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:52:02.087493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:52:02.087493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:52:02.087493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:52:02.087493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:52:02.087493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:02.087493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 161/326] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Oct 20 05:52:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:02.088134 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366df0d6-0923-4ee5-9936-5bbd1144fc09 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:02.118977 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 166/327] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:52:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 05:52:02.119694 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfb69e-0124-4af7-8de7-0db3544b7ee6 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:02.124319 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:02.140030 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:02.211415 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366df0d6-0923-4ee5-9936-5bbd1144fc09 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:02.231215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfb69e-0124-4af7-8de7-0db3544b7ee6 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:02.245270 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a1bc2-a541-4ba3-8a7d-283e7e9800b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:02.245882 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a1bc2-a541-4ba3-8a7d-283e7e9800b1 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:02.246159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a1bc2-a541-4ba3-8a7d-283e7e9800b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:02.246431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a1bc2-a541-4ba3-8a7d-283e7e9800b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:02.247157 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5a1bc2-a541-4ba3-8a7d-283e7e9800b1 None admin] RBAC: Adding query filter params (group.id=8cdaa84a20c045fe900b1d8633f92fc4, role.id=c1e0df53e1264ec79d91065e25acd29d, scope.project.id=e5ab74dfb5014b01bea8909bd1d64559) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:02.247463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5a1bc2-a541-4ba3-8a7d-283e7e9800b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:02.248932 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5a1bc2-a541-4ba3-8a7d-283e7e9800b1 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:02.254651 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 162/328] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Oct 20 05:52:02 2022] GET /identity/v3/role_assignments?group.id=8cdaa84a20c045fe900b1d8633f92fc4&scope.project.id=e5ab74dfb5014b01bea8909bd1d64559&role.id=c1e0df53e1264ec79d91065e25acd29d => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:02.257613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a1bc2-a541-4ba3-8a7d-283e7e9800b1 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:02.265068 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f8e89-36ae-490e-b860-5e30680414fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:02.265696 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f8e89-36ae-490e-b860-5e30680414fd None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:02.265969 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f8e89-36ae-490e-b860-5e30680414fd None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:02.266236 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f8e89-36ae-490e-b860-5e30680414fd None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:02.266905 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f8e89-36ae-490e-b860-5e30680414fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:02.267204 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f8e89-36ae-490e-b860-5e30680414fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:02.269252 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f8e89-36ae-490e-b860-5e30680414fd None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:02.283936 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 167/329] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:52:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:02.285495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f8e89-36ae-490e-b860-5e30680414fd None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:02.369542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:02.376470 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a1bc2-a541-4ba3-8a7d-283e7e9800b1 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:02.382844 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:02.408734 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f8e89-36ae-490e-b860-5e30680414fd None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:02.414562 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d7565-b5af-46de-b244-ce73c5b15e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:02.415217 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d7565-b5af-46de-b244-ce73c5b15e5c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:02.415494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d7565-b5af-46de-b244-ce73c5b15e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:02.415766 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d7565-b5af-46de-b244-ce73c5b15e5c None admin] PATH_INFO: `/v3/projects/e5ab74dfb5014b01bea8909bd1d64559` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:02.416971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d7565-b5af-46de-b244-ce73c5b15e5c None admin] RBAC: Authorizing `identity:get_project(project_id=e5ab74dfb5014b01bea8909bd1d64559)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:02.418421 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d7565-b5af-46de-b244-ce73c5b15e5c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:02.420301 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 163/330] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:52:02 2022] GET /identity/v3/projects/e5ab74dfb5014b01bea8909bd1d64559 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:02.425845 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d7565-b5af-46de-b244-ce73c5b15e5c None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:02.450028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05699d-af07-4f68-ba0c-edc9bc110df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:02.450743 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05699d-af07-4f68-ba0c-edc9bc110df5 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:02.451030 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05699d-af07-4f68-ba0c-edc9bc110df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:02.451302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05699d-af07-4f68-ba0c-edc9bc110df5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:02.529213 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05699d-af07-4f68-ba0c-edc9bc110df5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:02.530660 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05699d-af07-4f68-ba0c-edc9bc110df5 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:02.547734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d7565-b5af-46de-b244-ce73c5b15e5c None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:02.588382 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458fcc5-6c75-48d0-b171-cb201785f961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:02.589079 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458fcc5-6c75-48d0-b171-cb201785f961 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:02.589366 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458fcc5-6c75-48d0-b171-cb201785f961 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:02.589648 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458fcc5-6c75-48d0-b171-cb201785f961 None admin] PATH_INFO: `/v3/groups/8cdaa84a20c045fe900b1d8633f92fc4` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:02.590762 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9458fcc5-6c75-48d0-b171-cb201785f961 None admin] RBAC: Authorizing `identity:get_group(group_id=8cdaa84a20c045fe900b1d8633f92fc4)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:02.592257 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9458fcc5-6c75-48d0-b171-cb201785f961 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:02.594514 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 164/331] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Oct 20 05:52:02 2022] GET /identity/v3/groups/8cdaa84a20c045fe900b1d8633f92fc4 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:02.601903 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458fcc5-6c75-48d0-b171-cb201785f961 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:02.623233 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:02.638295 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:02.652893 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 168/332] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:52:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 05:52:02.658427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05699d-af07-4f68-ba0c-edc9bc110df5 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:02.709194 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458fcc5-6c75-48d0-b171-cb201785f961 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:02.741502 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6060b-6b00-490c-bda8-fe1ed587e6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:02.742101 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6060b-6b00-490c-bda8-fe1ed587e6ab None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:02.742374 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6060b-6b00-490c-bda8-fe1ed587e6ab None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:02.742641 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6060b-6b00-490c-bda8-fe1ed587e6ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:02.743351 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6060b-6b00-490c-bda8-fe1ed587e6ab None admin] RBAC: Adding query filter params (group.id=8cdaa84a20c045fe900b1d8633f92fc4, role.id=49dae0405c764c96b7bfbcf9d9efd011, scope.project.id=e5ab74dfb5014b01bea8909bd1d64559) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:02.743653 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6060b-6b00-490c-bda8-fe1ed587e6ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:02.745101 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6060b-6b00-490c-bda8-fe1ed587e6ab None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:02.750682 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 165/333] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Oct 20 05:52:02 2022] GET /identity/v3/role_assignments?group.id=8cdaa84a20c045fe900b1d8633f92fc4&scope.project.id=e5ab74dfb5014b01bea8909bd1d64559&role.id=49dae0405c764c96b7bfbcf9d9efd011 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:02.754250 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05699d-af07-4f68-ba0c-edc9bc110df5 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:02.792301 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab781206-fc6a-48b4-b60d-238defafd516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:02.792981 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab781206-fc6a-48b4-b60d-238defafd516 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:02.793263 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab781206-fc6a-48b4-b60d-238defafd516 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:02.793537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab781206-fc6a-48b4-b60d-238defafd516 None admin] PATH_INFO: `/v3/projects/edda8534137f405d818dbc74b7391c73` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:02.794721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab781206-fc6a-48b4-b60d-238defafd516 None admin] RBAC: Authorizing `identity:get_project(project_id=edda8534137f405d818dbc74b7391c73)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:02.796193 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab781206-fc6a-48b4-b60d-238defafd516 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:02.797909 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 169/334] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:52:02 2022] GET /identity/v3/projects/edda8534137f405d818dbc74b7391c73 => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:02.803781 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6060b-6b00-490c-bda8-fe1ed587e6ab None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:02.904312 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:02.911552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6060b-6b00-490c-bda8-fe1ed587e6ab None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:02.944423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707d492-daee-45e5-903d-898d0c4c4798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:02.945078 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707d492-daee-45e5-903d-898d0c4c4798 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:02.945356 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707d492-daee-45e5-903d-898d0c4c4798 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:02.945626 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707d492-daee-45e5-903d-898d0c4c4798 None admin] PATH_INFO: `/v3/groups/2e0ca34b71b34feaa4cc7634226e7232` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:02.946720 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8707d492-daee-45e5-903d-898d0c4c4798 None admin] RBAC: Authorizing `identity:get_group(group_id=2e0ca34b71b34feaa4cc7634226e7232)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:02.948156 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8707d492-daee-45e5-903d-898d0c4c4798 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:02.949778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 166/335] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Oct 20 05:52:02 2022] GET /identity/v3/groups/2e0ca34b71b34feaa4cc7634226e7232 => generated 229 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:02.955718 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab781206-fc6a-48b4-b60d-238defafd516 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:03.004584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:03.059510 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab781206-fc6a-48b4-b60d-238defafd516 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:03.093528 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8a7a4-b42c-41f8-8481-76b7b7aaab8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:03.094133 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8a7a4-b42c-41f8-8481-76b7b7aaab8f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:03.094410 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8a7a4-b42c-41f8-8481-76b7b7aaab8f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:03.094680 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8a7a4-b42c-41f8-8481-76b7b7aaab8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:03.095393 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b8a7a4-b42c-41f8-8481-76b7b7aaab8f None admin] RBAC: Adding query filter params (group.id=2e0ca34b71b34feaa4cc7634226e7232, role.id=d8debbb193834fac9cc25f81726f3a67, scope.project.id=edda8534137f405d818dbc74b7391c73) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:03.095702 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b8a7a4-b42c-41f8-8481-76b7b7aaab8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:03.097249 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b8a7a4-b42c-41f8-8481-76b7b7aaab8f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:03.105176 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 170/336] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Oct 20 05:52:02 2022] GET /identity/v3/role_assignments?group.id=2e0ca34b71b34feaa4cc7634226e7232&scope.project.id=edda8534137f405d818dbc74b7391c73&role.id=d8debbb193834fac9cc25f81726f3a67 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:05.597025 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9545e2-0637-48f9-80a3-356907add373 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:05.597999 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9545e2-0637-48f9-80a3-356907add373 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:05.598478 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9545e2-0637-48f9-80a3-356907add373 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:05.600271 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 167/337] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:05.605752 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1359cab-7ded-45e7-8be3-406753c556d4 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:05.606546 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1359cab-7ded-45e7-8be3-406753c556d4 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:05.607013 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1359cab-7ded-45e7-8be3-406753c556d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:05.633897 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-f1359cab-7ded-45e7-8be3-406753c556d4 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:05.635345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f1359cab-7ded-45e7-8be3-406753c556d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:05.723218 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 171/338] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:52:06.014519 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18c5de-9aa0-4d4e-ba4d-8791e246e730 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:06.014519 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18c5de-9aa0-4d4e-ba4d-8791e246e730 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:06.014773 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18c5de-9aa0-4d4e-ba4d-8791e246e730 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:06.016245 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 168/339] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:06.021789 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102946c-7c07-429f-a2c7-61798c5faae8 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:06.022072 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102946c-7c07-429f-a2c7-61798c5faae8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:06.022346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102946c-7c07-429f-a2c7-61798c5faae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:06.049611 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-c102946c-7c07-429f-a2c7-61798c5faae8 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:06.050824 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c102946c-7c07-429f-a2c7-61798c5faae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:06.059652 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18c5de-9aa0-4d4e-ba4d-8791e246e730 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:06.120678 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:06.186208 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 172/340] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:52:06.242350 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18c5de-9aa0-4d4e-ba4d-8791e246e730 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:06.284541 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05450f8-1889-49a8-9378-df415f74c816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:06.285863 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05450f8-1889-49a8-9378-df415f74c816 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:06.286359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05450f8-1889-49a8-9378-df415f74c816 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:06.286866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05450f8-1889-49a8-9378-df415f74c816 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:06.303831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05450f8-1889-49a8-9378-df415f74c816 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:06.305409 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05450f8-1889-49a8-9378-df415f74c816 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:06.329526 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-c05450f8-1889-49a8-9378-df415f74c816 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:06.329526 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:06.329526 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:06.329526 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:06.329526 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:06.329526 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:06.329526 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:06.329526 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:06.329526 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:06.329526 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:06.329526 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:06.329526 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:06.329526 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:06.329526 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:06.329526 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:06.329526 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:06.330548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:06.330548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:06.330548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:06.330548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:06.330548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:06.330548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:06.330548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:06.330548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:06.330548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:06.330548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:06.330548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:06.330548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:06.330548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:06.330548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:06.330548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:06.330548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:06.330548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:06.332565 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:06.332565 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:06.332565 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:06.332565 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 169/341] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:52:06.337271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102946c-7c07-429f-a2c7-61798c5faae8 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:06.350528 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1590fc8-c4e3-4deb-a65c-c91259f266ba None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:06.350872 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1590fc8-c4e3-4deb-a65c-c91259f266ba None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:06.351068 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1590fc8-c4e3-4deb-a65c-c91259f266ba None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:06.355253 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 170/342] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:52:06.358574 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d488a6b-2eec-4400-8cb3-1701d5d4c268 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:06.358852 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d488a6b-2eec-4400-8cb3-1701d5d4c268 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:06.359123 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d488a6b-2eec-4400-8cb3-1701d5d4c268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:06.395992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-2d488a6b-2eec-4400-8cb3-1701d5d4c268 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:06.397231 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2d488a6b-2eec-4400-8cb3-1701d5d4c268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:06.416534 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:06.485314 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102946c-7c07-429f-a2c7-61798c5faae8 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:06.486563 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 171/343] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 05:52:06.488626 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec6dde-0c23-4dca-a864-865830882e0d None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:06.488943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec6dde-0c23-4dca-a864-865830882e0d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:06.490598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec6dde-0c23-4dca-a864-865830882e0d None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:06.490598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 172/344] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:06.491133 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec6dde-0c23-4dca-a864-865830882e0d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:06.526269 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d0709-2053-4165-a828-7e2ce20511fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:06.526905 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d0709-2053-4165-a828-7e2ce20511fd None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:06.527182 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d0709-2053-4165-a828-7e2ce20511fd None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:06.527729 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d0709-2053-4165-a828-7e2ce20511fd None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:06.528487 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d0709-2053-4165-a828-7e2ce20511fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:06.528904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d0709-2053-4165-a828-7e2ce20511fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:06.530518 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d0709-2053-4165-a828-7e2ce20511fd None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:06.548301 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 173/345] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:06.550180 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7124469-287e-4c40-87bb-5373ba3760a2 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:06.550563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7124469-287e-4c40-87bb-5373ba3760a2 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:06.550929 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7124469-287e-4c40-87bb-5373ba3760a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:06.570254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d7124469-287e-4c40-87bb-5373ba3760a2 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:06.571575 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d7124469-287e-4c40-87bb-5373ba3760a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:06.596627 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:06.607399 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec6dde-0c23-4dca-a864-865830882e0d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:06.646111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 174/346] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:52:06.648641 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e19efe-9ae3-4eb0-acad-68a8e203ba23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:06.649368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e19efe-9ae3-4eb0-acad-68a8e203ba23 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:06.649705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e19efe-9ae3-4eb0-acad-68a8e203ba23 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:06.650034 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e19efe-9ae3-4eb0-acad-68a8e203ba23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:06.652820 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7124469-287e-4c40-87bb-5373ba3760a2 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:06.661090 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e19efe-9ae3-4eb0-acad-68a8e203ba23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:06.662589 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e19efe-9ae3-4eb0-acad-68a8e203ba23 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:06.669728 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:06.679423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-00e19efe-9ae3-4eb0-acad-68a8e203ba23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:06.679423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:06.679423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:06.679423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:06.679423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:06.679423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:06.679423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:06.679423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:06.679423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:06.679423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:06.679423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:06.679423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:06.679423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:06.679423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:06.679423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:06.679423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:06.681639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:06.681639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:06.681639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:06.681639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:06.681639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:06.681639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:06.681639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:06.681639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:06.681639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:06.681639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:06.681639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:06.681639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:06.681639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:06.681639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:06.681639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:06.681639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:06.681639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:06.683761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:06.683761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:06.683761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:06.683761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 173/347] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:06.690143 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e19efe-9ae3-4eb0-acad-68a8e203ba23 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:06.808690 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7124469-287e-4c40-87bb-5373ba3760a2 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:06.810310 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e19efe-9ae3-4eb0-acad-68a8e203ba23 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:06.845799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb97b20-ae22-4198-a150-c3036d7db0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:06.846431 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb97b20-ae22-4198-a150-c3036d7db0ba None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:06.847217 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb97b20-ae22-4198-a150-c3036d7db0ba None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:06.847486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb97b20-ae22-4198-a150-c3036d7db0ba None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:06.848154 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb97b20-ae22-4198-a150-c3036d7db0ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:06.849744 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb97b20-ae22-4198-a150-c3036d7db0ba None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:06.853856 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648ae36-14ce-4509-8dc1-315dface34cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:06.854706 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648ae36-14ce-4509-8dc1-315dface34cc None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:06.855184 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648ae36-14ce-4509-8dc1-315dface34cc None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:06.855658 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648ae36-14ce-4509-8dc1-315dface34cc None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:06.856502 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648ae36-14ce-4509-8dc1-315dface34cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:06.857025 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648ae36-14ce-4509-8dc1-315dface34cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:06.859385 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648ae36-14ce-4509-8dc1-315dface34cc None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:06.872578 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 175/348] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Oct 20 05:52:06 2022] POST /identity/v3/users => generated 246 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:52:06.873493 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb97b20-ae22-4198-a150-c3036d7db0ba None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:06.880685 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 174/349] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:06.882377 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648ae36-14ce-4509-8dc1-315dface34cc None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:06.905100 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:06.950462 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:06.998425 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648ae36-14ce-4509-8dc1-315dface34cc None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:07.007045 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb97b20-ae22-4198-a150-c3036d7db0ba None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:07.038374 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8b3f0-5860-4f23-9394-d790892905c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:07.039416 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8b3f0-5860-4f23-9394-d790892905c5 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:07.039902 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8b3f0-5860-4f23-9394-d790892905c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:07.040378 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8b3f0-5860-4f23-9394-d790892905c5 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:07.041298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a8b3f0-5860-4f23-9394-d790892905c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:07.042980 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a8b3f0-5860-4f23-9394-d790892905c5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:07.068998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb9817-3963-45ce-9ba1-d3b8579b78d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:07.069573 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb9817-3963-45ce-9ba1-d3b8579b78d8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:07.069850 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb9817-3963-45ce-9ba1-d3b8579b78d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:07.070120 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb9817-3963-45ce-9ba1-d3b8579b78d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:07.077011 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 175/350] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Oct 20 05:52:06 2022] POST /identity/v3/users => generated 248 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 05:52:07.077011 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8b3f0-5860-4f23-9394-d790892905c5 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:07.090237 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb9817-3963-45ce-9ba1-d3b8579b78d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:07.091799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb9817-3963-45ce-9ba1-d3b8579b78d8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:07.112288 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-8eeb9817-3963-45ce-9ba1-d3b8579b78d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:07.112288 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:07.112288 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:07.112288 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:07.112288 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:07.112288 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:07.112288 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:07.112288 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:07.112288 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:07.112288 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:07.112288 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:07.112288 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:07.112288 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:07.112288 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:07.112288 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:07.112288 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:07.113304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:07.113304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:07.113304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:07.113304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:07.113304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:07.113304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:07.113304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:07.113304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:07.113304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:07.113304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:07.113304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:07.113304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:07.113304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:07.113304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:07.113304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:07.113304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:07.113304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:07.114166 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:07.114166 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:07.114166 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:07.116997 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 176/351] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:07.126117 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb9817-3963-45ce-9ba1-d3b8579b78d8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:07.225092 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:07.235656 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8b3f0-5860-4f23-9394-d790892905c5 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:07.255323 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:07.261138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb9817-3963-45ce-9ba1-d3b8579b78d8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:07.279392 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d049a-8074-4766-afe6-9d5063f2773a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:07.280122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d049a-8074-4766-afe6-9d5063f2773a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:07.280404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d049a-8074-4766-afe6-9d5063f2773a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:07.280677 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d049a-8074-4766-afe6-9d5063f2773a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:07.293524 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d049a-8074-4766-afe6-9d5063f2773a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:07.295317 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d049a-8074-4766-afe6-9d5063f2773a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:07.317901 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76849492-9df7-404b-bf5d-669d835352d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:07.318551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76849492-9df7-404b-bf5d-669d835352d0 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:07.318827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76849492-9df7-404b-bf5d-669d835352d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:07.319151 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76849492-9df7-404b-bf5d-669d835352d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:07.319757 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76849492-9df7-404b-bf5d-669d835352d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:07.320040 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76849492-9df7-404b-bf5d-669d835352d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:07.320442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-6e4d049a-8074-4766-afe6-9d5063f2773a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:07.320442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:07.320442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:07.320442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:07.320442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:07.320442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:07.320442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:07.320442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:07.320442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:07.320442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:07.320442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:07.320442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:07.320442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:07.320442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:07.320442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:07.320442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:07.321338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:07.321338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:07.321338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:07.321338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:07.321338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:07.321338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:07.321338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:07.321338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:07.321338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:07.321338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:07.321338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:07.321338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:07.321338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:07.321338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:07.321338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:07.321338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:07.321338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:07.322163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:07.322163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:07.322163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:07.322334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76849492-9df7-404b-bf5d-669d835352d0 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:07.322408 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 176/352] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:07.329352 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d049a-8074-4766-afe6-9d5063f2773a None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:07.335726 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 177/353] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:07.341568 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76849492-9df7-404b-bf5d-669d835352d0 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:07.468910 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d049a-8074-4766-afe6-9d5063f2773a None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:07.474358 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76849492-9df7-404b-bf5d-669d835352d0 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:07.514617 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ec6e7-581d-469f-8689-80cca349421b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:07.515233 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ec6e7-581d-469f-8689-80cca349421b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:07.515503 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ec6e7-581d-469f-8689-80cca349421b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:07.515769 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ec6e7-581d-469f-8689-80cca349421b None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:07.516405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ec6e7-581d-469f-8689-80cca349421b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:07.516729 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ec6e7-581d-469f-8689-80cca349421b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:07.518074 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ec6e7-581d-469f-8689-80cca349421b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:07.521144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:07.529595 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2256dea-e900-41ae-b838-ef0bd843328b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:07.530201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2256dea-e900-41ae-b838-ef0bd843328b None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:07.530479 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2256dea-e900-41ae-b838-ef0bd843328b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:07.530749 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2256dea-e900-41ae-b838-ef0bd843328b None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:07.531387 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2256dea-e900-41ae-b838-ef0bd843328b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:07.533452 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2256dea-e900-41ae-b838-ef0bd843328b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:07.538084 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 177/354] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:07.539582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d2e49-df48-4369-b7d4-8943adc6b8da None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:07.540073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d2e49-df48-4369-b7d4-8943adc6b8da None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:07.540539 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d2e49-df48-4369-b7d4-8943adc6b8da None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:07.542288 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 178/355] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:07.546316 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d2e49-df48-4369-b7d4-8943adc6b8da None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:07.564603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 178/356] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Oct 20 05:52:07 2022] POST /identity/v3/users => generated 248 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 05:52:07.568745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d62f08-05d8-44e2-a16c-143aa5458057 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:07.568745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d62f08-05d8-44e2-a16c-143aa5458057 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:07.568745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d62f08-05d8-44e2-a16c-143aa5458057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:07.609507 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:07.611465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-24d62f08-05d8-44e2-a16c-143aa5458057 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:07.614268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-24d62f08-05d8-44e2-a16c-143aa5458057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:07.707109 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d2e49-df48-4369-b7d4-8943adc6b8da None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:07.745830 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c29d0c-ceed-4cfd-84ee-7cf6cd8b19e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:07.746455 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c29d0c-ceed-4cfd-84ee-7cf6cd8b19e7 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:07.746729 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c29d0c-ceed-4cfd-84ee-7cf6cd8b19e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:07.747004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c29d0c-ceed-4cfd-84ee-7cf6cd8b19e7 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:07.747641 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c29d0c-ceed-4cfd-84ee-7cf6cd8b19e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:07.749337 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c29d0c-ceed-4cfd-84ee-7cf6cd8b19e7 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:07.779043 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 179/357] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:52:07.788750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 179/358] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Oct 20 05:52:07 2022] POST /identity/v3/users => generated 249 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:52:08.063468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d62f08-05d8-44e2-a16c-143aa5458057 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:08.192860 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d62f08-05d8-44e2-a16c-143aa5458057 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:08.215681 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:08.245634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fdb93-899a-4c8e-bedd-24ac0f3fd661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:08.246258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fdb93-899a-4c8e-bedd-24ac0f3fd661 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:08.246536 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fdb93-899a-4c8e-bedd-24ac0f3fd661 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:08.246810 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fdb93-899a-4c8e-bedd-24ac0f3fd661 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:08.247476 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448fdb93-899a-4c8e-bedd-24ac0f3fd661 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:08.248887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448fdb93-899a-4c8e-bedd-24ac0f3fd661 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:08.257986 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.sql.core [None req-448fdb93-899a-4c8e-bedd-24ac0f3fd661 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 20 05:52:08.257986 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 20 05:52:08.257986 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [parameters: {'id': '6bfbac90b2f8414b94678c920ae5824a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 20 05:52:08.257986 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81354) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 20 05:52:08.275384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-448fdb93-899a-4c8e-bedd-24ac0f3fd661 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 20 05:52:08.275384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:08.275384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 05:52:08.275384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 05:52:08.275384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 05:52:08.275384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 05:52:08.275384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 05:52:08.275384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 05:52:08.275384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 05:52:08.275384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application conn.query(q) Oct 20 05:52:08.275384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 05:52:08.275384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 05:52:08.275384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 05:52:08.275384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application result.read() Oct 20 05:52:08.275384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 05:52:08.276272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 05:52:08.276272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 05:52:08.276272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 05:52:08.276272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 05:52:08.276272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 05:52:08.276272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 05:52:08.276272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 05:52:08.276272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 20 05:52:08.276272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:08.276272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 20 05:52:08.276272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:08.276272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:08.276272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 20 05:52:08.276272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 20 05:52:08.276272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 20 05:52:08.276272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return ref.to_dict() Oct 20 05:52:08.276272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 05:52:08.276272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application next(self.gen) Oct 20 05:52:08.277215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 20 05:52:08.277215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application yield resource Oct 20 05:52:08.277215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 05:52:08.277215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application next(self.gen) Oct 20 05:52:08.277215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 20 05:52:08.277215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.session.rollback() Oct 20 05:52:08.277215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 20 05:52:08.277215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.force_reraise() Oct 20 05:52:08.277215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 20 05:52:08.277215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise self.value Oct 20 05:52:08.277215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 20 05:52:08.277215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 20 05:52:08.277215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 20 05:52:08.277215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application session.commit() Oct 20 05:52:08.277215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 20 05:52:08.277215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 20 05:52:08.277215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 20 05:52:08.278065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._prepare_impl() Oct 20 05:52:08.278065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 20 05:52:08.278065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.session.flush() Oct 20 05:52:08.278065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 20 05:52:08.278065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._flush(objects) Oct 20 05:52:08.278065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 20 05:52:08.278065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 20 05:52:08.278065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 20 05:52:08.278065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application compat.raise_( Oct 20 05:52:08.278065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 05:52:08.278065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception Oct 20 05:52:08.278065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 20 05:52:08.278065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application flush_context.execute() Oct 20 05:52:08.278065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 20 05:52:08.278065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rec.execute(self) Oct 20 05:52:08.278065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 20 05:52:08.278065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 20 05:52:08.278914 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 20 05:52:08.278914 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 20 05:52:08.278914 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 20 05:52:08.278914 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 20 05:52:08.278914 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 20 05:52:08.278914 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 20 05:52:08.278914 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 20 05:52:08.278914 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 20 05:52:08.278914 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 20 05:52:08.278914 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 20 05:52:08.278914 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 20 05:52:08.278914 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 20 05:52:08.278914 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 20 05:52:08.278914 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 20 05:52:08.278914 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 05:52:08.278914 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception Oct 20 05:52:08.279686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 05:52:08.279686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 05:52:08.279686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 05:52:08.279686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 05:52:08.279686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 05:52:08.279686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 05:52:08.279686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 05:52:08.279686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application conn.query(q) Oct 20 05:52:08.279686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 05:52:08.279686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 05:52:08.279686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 05:52:08.279686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application result.read() Oct 20 05:52:08.279686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 05:52:08.279686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 05:52:08.279686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 05:52:08.279686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 05:52:08.280457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 05:52:08.280457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 05:52:08.280457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 05:52:08.280457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 05:52:08.280457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 20 05:52:08.280457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 20 05:52:08.280457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [parameters: {'id': '6bfbac90b2f8414b94678c920ae5824a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 20 05:52:08.280457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 20 05:52:08.280457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:08.280457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:52:08.280457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:08.280457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:08.280457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:08.280457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:08.280457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:08.280457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:08.283140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:08.283140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:08.283140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:08.283140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:08.283140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:08.283140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:08.283140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 20 05:52:08.283140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 20 05:52:08.283140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:08.283140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:08.283140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 20 05:52:08.283140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 20 05:52:08.283140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 20 05:52:08.283140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 20 05:52:08.283140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 20 05:52:08.283140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:08.285227 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 180/359] 10.222.0.20 () {60 vars in 1131 bytes} [Thu Oct 20 05:52:08 2022] POST /identity/v3/roles => generated 144 bytes in 217 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:08.285439 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c29d0c-ceed-4cfd-84ee-7cf6cd8b19e7 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:08.372107 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:08.394887 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c29d0c-ceed-4cfd-84ee-7cf6cd8b19e7 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:08.429923 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79a222-8c27-4cb8-bc09-1c8c5db3ca0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:08.430569 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79a222-8c27-4cb8-bc09-1c8c5db3ca0b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:08.430848 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79a222-8c27-4cb8-bc09-1c8c5db3ca0b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:08.431121 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79a222-8c27-4cb8-bc09-1c8c5db3ca0b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:08.442399 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79a222-8c27-4cb8-bc09-1c8c5db3ca0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:52:08.447725 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-3d79a222-8c27-4cb8-bc09-1c8c5db3ca0b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 05:52:08.447725 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:08.447725 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:08.447725 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:08.447725 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:08.447725 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:08.447725 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:08.447725 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:08.447725 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:08.447725 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:08.447725 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:08.447725 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:08.447725 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:52:08.447725 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:52:08.447725 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:52:08.447725 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:52:08.449147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:52:08.449147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:52:08.449147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:52:08.449147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:52:08.449147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:08.449147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:08.449147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:08.449147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:08.449147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:08.449147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:08.449147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:08.449147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:08.449147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:08.449147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:08.449147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:08.449147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:08.451022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:08.451022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:08.451022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:08.451022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:08.451022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:52:08.451022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:52:08.451022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:52:08.451022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:52:08.451022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:52:08.451022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:52:08.451022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 05:52:08.451022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:08.455816 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 180/360] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Oct 20 05:52:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:08.456553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fdb93-899a-4c8e-bedd-24ac0f3fd661 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:08.560633 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fdb93-899a-4c8e-bedd-24ac0f3fd661 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:08.596470 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272c861-c148-4b24-b28b-dcecb6597d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:08.597147 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272c861-c148-4b24-b28b-dcecb6597d0a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:08.597425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272c861-c148-4b24-b28b-dcecb6597d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:08.597699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272c861-c148-4b24-b28b-dcecb6597d0a None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:08.598360 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5272c861-c148-4b24-b28b-dcecb6597d0a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:08.598665 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5272c861-c148-4b24-b28b-dcecb6597d0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:08.600043 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5272c861-c148-4b24-b28b-dcecb6597d0a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:08.612833 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 181/361] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:09.700434 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011c501-a698-4aed-a74c-dc0e53c4f6b3 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:09.701004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011c501-a698-4aed-a74c-dc0e53c4f6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:09.701004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011c501-a698-4aed-a74c-dc0e53c4f6b3 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:09.702598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 181/362] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:09.711473 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67dd49-aa1c-4b03-acf2-3f01571ec8dd None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:09.712319 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67dd49-aa1c-4b03-acf2-3f01571ec8dd None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:09.712869 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67dd49-aa1c-4b03-acf2-3f01571ec8dd None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:09.714229 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415befe-e196-4d4c-9ca5-548ca7f9c49c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:09.714511 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415befe-e196-4d4c-9ca5-548ca7f9c49c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:09.714782 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415befe-e196-4d4c-9ca5-548ca7f9c49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:09.715050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 182/363] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:09.722962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254a550-293d-46f8-a7ec-e0c3f916cd53 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:09.723237 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254a550-293d-46f8-a7ec-e0c3f916cd53 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:09.723503 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254a550-293d-46f8-a7ec-e0c3f916cd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:09.746716 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-e415befe-e196-4d4c-9ca5-548ca7f9c49c None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:09.747929 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e415befe-e196-4d4c-9ca5-548ca7f9c49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:09.760306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-2254a550-293d-46f8-a7ec-e0c3f916cd53 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:09.761493 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2254a550-293d-46f8-a7ec-e0c3f916cd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:09.812236 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:09.869848 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 183/364] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:52:09.871750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 182/365] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 05:52:10.115796 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ec489-daaf-404b-94b6-884295cf6fb7 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:10.116137 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ec489-daaf-404b-94b6-884295cf6fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:10.116246 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ec489-daaf-404b-94b6-884295cf6fb7 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:10.118145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 184/366] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:10.123137 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee313200-08bb-4acf-beb8-72700330bd73 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:10.123421 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee313200-08bb-4acf-beb8-72700330bd73 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:10.123693 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee313200-08bb-4acf-beb8-72700330bd73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:10.149054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-ee313200-08bb-4acf-beb8-72700330bd73 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:10.150277 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ee313200-08bb-4acf-beb8-72700330bd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:10.161018 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ec489-daaf-404b-94b6-884295cf6fb7 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:10.270736 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 183/367] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:52:10.271703 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee313200-08bb-4acf-beb8-72700330bd73 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:10.277120 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:10.341221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ec489-daaf-404b-94b6-884295cf6fb7 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:10.405934 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812af010-9b35-4b3a-b2c8-95acd57cee28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:10.406569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812af010-9b35-4b3a-b2c8-95acd57cee28 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:10.406843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812af010-9b35-4b3a-b2c8-95acd57cee28 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:10.407983 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812af010-9b35-4b3a-b2c8-95acd57cee28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:10.427946 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812af010-9b35-4b3a-b2c8-95acd57cee28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:52:10.435661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-812af010-9b35-4b3a-b2c8-95acd57cee28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:52:10.435661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:10.435661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:10.435661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:10.435661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:10.435661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:10.435661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:10.435661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:10.435661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:10.435661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:10.435661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:10.435661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:10.435661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:52:10.435661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:52:10.435661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:52:10.435661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:52:10.436524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:52:10.436524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:52:10.436524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:52:10.436524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:52:10.436524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:10.436524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:10.436524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:10.436524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:10.436524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:10.436524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:10.436524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:10.436524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:10.436524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:10.436524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:10.436524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:10.436524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:10.437362 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:10.437362 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:10.437362 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:10.437362 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:10.437362 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:52:10.437362 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:52:10.437362 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:52:10.437362 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:52:10.437362 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:52:10.437362 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:52:10.437362 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:52:10.437362 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:10.438917 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 185/368] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Oct 20 05:52:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:52:10.445077 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee313200-08bb-4acf-beb8-72700330bd73 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:10.447157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812af010-9b35-4b3a-b2c8-95acd57cee28 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:10.516191 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:10.517052 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfdc501-dbac-436f-899a-cc05f0e5fcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:10.517695 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfdc501-dbac-436f-899a-cc05f0e5fcd8 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:10.517968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfdc501-dbac-436f-899a-cc05f0e5fcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:10.518232 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfdc501-dbac-436f-899a-cc05f0e5fcd8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:10.542736 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfdc501-dbac-436f-899a-cc05f0e5fcd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:52:10.549060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-abfdc501-dbac-436f-899a-cc05f0e5fcd8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:52:10.549060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:10.549060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:10.549060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:10.549060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:10.549060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:10.549060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:10.549060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:10.549060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:10.549060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:10.549060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:10.549060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:10.549060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:52:10.549060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:52:10.549060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:52:10.549060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:52:10.549995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:52:10.549995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:52:10.549995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:52:10.549995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:52:10.549995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:10.549995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:10.549995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:10.549995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:10.549995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:10.549995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:10.549995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:10.549995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:10.549995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:10.549995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:10.549995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:10.549995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:10.550886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:10.550886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:10.550886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:10.550886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:10.550886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:52:10.550886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:52:10.550886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:52:10.550886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:52:10.550886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:52:10.550886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:52:10.550886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:52:10.550886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:10.550886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 184/369] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Oct 20 05:52:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:10.556280 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfdc501-dbac-436f-899a-cc05f0e5fcd8 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:10.618479 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812af010-9b35-4b3a-b2c8-95acd57cee28 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:10.629513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:10.662054 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5036329-22f9-489d-af3c-63578b0c97a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:10.662919 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5036329-22f9-489d-af3c-63578b0c97a1 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:10.663093 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5036329-22f9-489d-af3c-63578b0c97a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:10.663226 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5036329-22f9-489d-af3c-63578b0c97a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:10.663969 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5036329-22f9-489d-af3c-63578b0c97a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:10.664179 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5036329-22f9-489d-af3c-63578b0c97a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:10.665813 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5036329-22f9-489d-af3c-63578b0c97a1 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:10.672004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfdc501-dbac-436f-899a-cc05f0e5fcd8 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:10.675316 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 186/370] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Oct 20 05:52:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:10.676215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5036329-22f9-489d-af3c-63578b0c97a1 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:10.719334 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112dad6-c085-463c-9ecf-2e8ea0a26ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:10.720945 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112dad6-c085-463c-9ecf-2e8ea0a26ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:10.721280 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112dad6-c085-463c-9ecf-2e8ea0a26ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:10.722140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112dad6-c085-463c-9ecf-2e8ea0a26ca2 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:10.722837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d112dad6-c085-463c-9ecf-2e8ea0a26ca2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:10.723213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d112dad6-c085-463c-9ecf-2e8ea0a26ca2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:10.724632 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d112dad6-c085-463c-9ecf-2e8ea0a26ca2 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:10.736864 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 185/371] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Oct 20 05:52:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:10.737808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112dad6-c085-463c-9ecf-2e8ea0a26ca2 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:10.793089 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:10.805740 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5036329-22f9-489d-af3c-63578b0c97a1 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:10.841072 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20cab70-e61d-44e4-9b22-040fb148c996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:10.842133 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20cab70-e61d-44e4-9b22-040fb148c996 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:10.842604 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20cab70-e61d-44e4-9b22-040fb148c996 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:10.843063 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20cab70-e61d-44e4-9b22-040fb148c996 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:10.853981 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20cab70-e61d-44e4-9b22-040fb148c996 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:52:10.859322 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112dad6-c085-463c-9ecf-2e8ea0a26ca2 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:10.859675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-a20cab70-e61d-44e4-9b22-040fb148c996 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:52:10.859675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:10.859675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:10.859675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:10.859675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:10.859675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:10.859675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:10.859675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:10.859675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:10.859675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:10.859675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:10.859675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:10.859675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:52:10.859675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:52:10.859675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:52:10.859675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:52:10.860558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:52:10.860558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:52:10.860558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:52:10.860558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:52:10.860558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:10.860558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:10.860558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:10.860558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:10.860558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:10.860558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:10.860558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:10.860558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:10.860558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:10.860558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:10.860558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:10.860558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:10.861413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:10.861413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:10.861413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:10.861413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:10.861413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:52:10.861413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:52:10.861413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:52:10.861413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:52:10.861413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:52:10.861413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:52:10.861413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:52:10.861413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:10.862672 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 187/372] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Oct 20 05:52:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:10.863341 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20cab70-e61d-44e4-9b22-040fb148c996 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:10.887196 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:10.899845 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c4a7d-4cf2-4b9a-8697-84880c1cb6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:10.900478 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c4a7d-4cf2-4b9a-8697-84880c1cb6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:10.900773 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c4a7d-4cf2-4b9a-8697-84880c1cb6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:10.901047 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c4a7d-4cf2-4b9a-8697-84880c1cb6a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:10.912153 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c4a7d-4cf2-4b9a-8697-84880c1cb6a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:10.913707 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c4a7d-4cf2-4b9a-8697-84880c1cb6a5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:10.929866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-555c4a7d-4cf2-4b9a-8697-84880c1cb6a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:10.929866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:10.929866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:10.929866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:10.929866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:10.929866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:10.929866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:10.929866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:10.929866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:10.929866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:10.929866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:10.929866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:10.929866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:10.929866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:10.929866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:10.929866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:10.930934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:10.930934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:10.930934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:10.930934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:10.930934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:10.930934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:10.930934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:10.930934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:10.930934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:10.930934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:10.930934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:10.930934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:10.930934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:10.930934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:10.930934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:10.930934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:10.930934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:10.931758 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:10.931758 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:10.931758 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:10.936233 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 186/373] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:10.937773 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818232df-b190-47fb-bdbc-175fa0a20a34 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:10.938089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818232df-b190-47fb-bdbc-175fa0a20a34 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:10.938376 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818232df-b190-47fb-bdbc-175fa0a20a34 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:10.939797 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 187/374] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:10.940423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818232df-b190-47fb-bdbc-175fa0a20a34 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:10.971702 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20cab70-e61d-44e4-9b22-040fb148c996 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:11.004015 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c6907-c31f-49cd-b09f-9f204862ac4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:11.004639 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c6907-c31f-49cd-b09f-9f204862ac4d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:11.004957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c6907-c31f-49cd-b09f-9f204862ac4d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:11.005228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c6907-c31f-49cd-b09f-9f204862ac4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:11.015862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749c6907-c31f-49cd-b09f-9f204862ac4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:11.018111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749c6907-c31f-49cd-b09f-9f204862ac4d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:11.033867 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-749c6907-c31f-49cd-b09f-9f204862ac4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:11.033867 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:11.033867 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:11.033867 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:11.033867 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:11.033867 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:11.033867 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:11.033867 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:11.033867 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:11.033867 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:11.033867 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:11.033867 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:11.033867 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:11.033867 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:11.033867 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:11.033867 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:11.035641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:11.035641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:11.035641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:11.035641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:11.035641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:11.035641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:11.035641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:11.035641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:11.035641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:11.035641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:11.035641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:11.035641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:11.035641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:11.035641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:11.035641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:11.035641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:11.035641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:11.036660 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:11.036660 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:11.036660 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:11.036660 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 188/375] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:11.036660 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c6907-c31f-49cd-b09f-9f204862ac4d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:11.039637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:11.053122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818232df-b190-47fb-bdbc-175fa0a20a34 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:11.088103 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f6766-fc6d-479e-9904-1cc462da8db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:11.088722 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f6766-fc6d-479e-9904-1cc462da8db1 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:11.089013 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f6766-fc6d-479e-9904-1cc462da8db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:11.089280 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f6766-fc6d-479e-9904-1cc462da8db1 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:11.089930 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f6766-fc6d-479e-9904-1cc462da8db1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:11.090229 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f6766-fc6d-479e-9904-1cc462da8db1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:11.091600 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f6766-fc6d-479e-9904-1cc462da8db1 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:11.104816 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 188/376] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Oct 20 05:52:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:11.104816 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3d7cf-03d1-4ad9-8eb3-d02fed812cd6 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:11.104816 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3d7cf-03d1-4ad9-8eb3-d02fed812cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:11.104816 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3d7cf-03d1-4ad9-8eb3-d02fed812cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:11.121083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-faa3d7cf-03d1-4ad9-8eb3-d02fed812cd6 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:11.122369 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-faa3d7cf-03d1-4ad9-8eb3-d02fed812cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:11.144395 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c6907-c31f-49cd-b09f-9f204862ac4d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:11.154630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:11.181235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef182692-1687-4744-8bfc-a308a8b44797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:11.181754 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef182692-1687-4744-8bfc-a308a8b44797 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:11.182030 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef182692-1687-4744-8bfc-a308a8b44797 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:11.182296 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef182692-1687-4744-8bfc-a308a8b44797 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:11.182920 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef182692-1687-4744-8bfc-a308a8b44797 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:11.183222 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef182692-1687-4744-8bfc-a308a8b44797 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:11.184582 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef182692-1687-4744-8bfc-a308a8b44797 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:11.196959 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 189/377] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:11.201534 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef182692-1687-4744-8bfc-a308a8b44797 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:11.202557 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 189/378] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:52:11.209035 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3d7cf-03d1-4ad9-8eb3-d02fed812cd6 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:11.287207 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:11.329906 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef182692-1687-4744-8bfc-a308a8b44797 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:11.335026 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3d7cf-03d1-4ad9-8eb3-d02fed812cd6 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:11.373691 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099b6bf-4933-411b-8035-033c72c4c06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:11.374327 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099b6bf-4933-411b-8035-033c72c4c06f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:11.374600 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099b6bf-4933-411b-8035-033c72c4c06f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:11.374866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099b6bf-4933-411b-8035-033c72c4c06f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:11.376840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf94b6d-b6c5-42ab-89f7-e3d00bdd2461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:11.378377 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf94b6d-b6c5-42ab-89f7-e3d00bdd2461 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:11.378699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf94b6d-b6c5-42ab-89f7-e3d00bdd2461 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:11.378971 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf94b6d-b6c5-42ab-89f7-e3d00bdd2461 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:11.379612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf94b6d-b6c5-42ab-89f7-e3d00bdd2461 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:11.379910 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf94b6d-b6c5-42ab-89f7-e3d00bdd2461 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:11.381302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf94b6d-b6c5-42ab-89f7-e3d00bdd2461 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:11.386441 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099b6bf-4933-411b-8035-033c72c4c06f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:11.387870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099b6bf-4933-411b-8035-033c72c4c06f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:11.396359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 190/379] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:11.397737 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf94b6d-b6c5-42ab-89f7-e3d00bdd2461 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:11.404559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-f099b6bf-4933-411b-8035-033c72c4c06f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:11.404559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:11.404559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:11.404559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:11.404559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:11.404559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:11.404559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:11.404559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:11.404559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:11.404559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:11.404559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:11.404559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:11.404559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:11.404559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:11.404559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:11.404559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:11.405457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:11.405457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:11.405457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:11.405457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:11.405457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:11.405457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:11.405457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:11.405457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:11.405457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:11.405457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:11.405457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:11.405457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:11.405457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:11.405457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:11.405457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:11.405457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:11.405457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:11.406305 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:11.406305 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:11.406305 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:11.406305 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 190/380] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:11.406913 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099b6bf-4933-411b-8035-033c72c4c06f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:11.430423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:11.513817 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099b6bf-4933-411b-8035-033c72c4c06f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:11.521693 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf94b6d-b6c5-42ab-89f7-e3d00bdd2461 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:11.548994 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:11.554821 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05655404-04e8-4c3b-bbc0-9e27ab5ae42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:11.555491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05655404-04e8-4c3b-bbc0-9e27ab5ae42c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:11.555746 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05655404-04e8-4c3b-bbc0-9e27ab5ae42c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:11.556012 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05655404-04e8-4c3b-bbc0-9e27ab5ae42c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:11.563979 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18dfdb-229d-4318-bb07-48e0ebdc0532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:11.564634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18dfdb-229d-4318-bb07-48e0ebdc0532 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:11.564958 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18dfdb-229d-4318-bb07-48e0ebdc0532 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:11.565274 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18dfdb-229d-4318-bb07-48e0ebdc0532 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:11.565850 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05655404-04e8-4c3b-bbc0-9e27ab5ae42c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:11.567469 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05655404-04e8-4c3b-bbc0-9e27ab5ae42c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:11.573922 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e18dfdb-229d-4318-bb07-48e0ebdc0532 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:11.575622 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e18dfdb-229d-4318-bb07-48e0ebdc0532 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:11.580587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-05655404-04e8-4c3b-bbc0-9e27ab5ae42c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 20 05:52:11.580587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:11.580587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:11.580587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:11.580587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:11.580587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:11.580587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:11.580587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:11.580587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:11.580587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:11.580587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:11.580587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:11.580587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:52:11.580587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:52:11.580587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:52:11.580587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:52:11.581691 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:11.581691 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:11.581691 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:52:11.581691 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:11.581691 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:52:11.581691 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:11.581691 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:11.581691 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:11.581691 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:11.581691 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:11.581691 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:11.581691 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:11.581691 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:11.581691 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:11.581691 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:11.581691 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:11.581691 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:11.581691 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:11.582587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:11.582587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:11.582587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:52:11.582587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:52:11.582587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:52:11.582587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:52:11.582587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:52:11.582587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:52:11.582587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 20 05:52:11.582587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:11.582587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 191/381] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Oct 20 05:52:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:11.582587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05655404-04e8-4c3b-bbc0-9e27ab5ae42c None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:11.592457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-5e18dfdb-229d-4318-bb07-48e0ebdc0532 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 20 05:52:11.592457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:11.592457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:11.592457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:11.592457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:11.592457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:11.592457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:11.592457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:11.592457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:11.592457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:11.592457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:11.592457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:11.592457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:52:11.592457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:52:11.592457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:52:11.592457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:52:11.593408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:11.593408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:11.593408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:52:11.593408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:11.593408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:52:11.593408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:11.593408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:11.593408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:11.593408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:11.593408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:11.593408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:11.593408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:11.593408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:11.593408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:11.593408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:11.593408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:11.593408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:11.593408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:11.594271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:11.594271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:11.594271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:52:11.594271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:52:11.594271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:52:11.594271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:52:11.594271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:52:11.594271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:52:11.594271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 20 05:52:11.594271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:11.597375 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 191/382] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:52:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:11.598944 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18dfdb-229d-4318-bb07-48e0ebdc0532 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:11.675859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:11.694909 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05655404-04e8-4c3b-bbc0-9e27ab5ae42c None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:11.699893 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18dfdb-229d-4318-bb07-48e0ebdc0532 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:11.727923 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2a001-2453-4bec-9639-df97a351d343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:11.728547 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2a001-2453-4bec-9639-df97a351d343 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:11.728810 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2a001-2453-4bec-9639-df97a351d343 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:11.729079 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2a001-2453-4bec-9639-df97a351d343 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:11.729705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d2a001-2453-4bec-9639-df97a351d343 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:11.730003 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d2a001-2453-4bec-9639-df97a351d343 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:11.731369 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d2a001-2453-4bec-9639-df97a351d343 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:11.733107 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5b34c-2a94-4067-af03-ad8d0d999ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:11.733726 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5b34c-2a94-4067-af03-ad8d0d999ecf None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:11.734001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5b34c-2a94-4067-af03-ad8d0d999ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:11.734268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5b34c-2a94-4067-af03-ad8d0d999ecf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:11.742977 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 192/383] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:11.743763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2a001-2453-4bec-9639-df97a351d343 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:11.748257 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5b34c-2a94-4067-af03-ad8d0d999ecf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:52:11.753592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-6dc5b34c-2a94-4067-af03-ad8d0d999ecf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:52:11.753592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:11.753592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:11.753592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:11.753592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:11.753592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:11.753592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:11.753592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:11.753592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:11.753592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:11.753592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:11.753592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:11.753592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:52:11.753592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:52:11.753592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:52:11.753592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:52:11.754481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:52:11.754481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:52:11.754481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:52:11.754481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:52:11.754481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:11.754481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:11.754481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:11.754481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:11.754481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:11.754481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:11.754481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:11.754481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:11.754481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:11.754481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:11.754481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:11.754481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:11.755266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:11.755266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:11.755266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:11.755266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:11.755266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:52:11.755266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:52:11.755266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:52:11.755266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:52:11.755266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:52:11.755266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:52:11.755266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:52:11.755266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:11.755266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 192/384] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Oct 20 05:52:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:11.755910 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5b34c-2a94-4067-af03-ad8d0d999ecf None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:11.802290 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:11.844173 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2a001-2453-4bec-9639-df97a351d343 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:11.855991 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5b34c-2a94-4067-af03-ad8d0d999ecf None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:11.879349 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413dd9b-782c-486c-9b9e-e13cfb6e050d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:11.880175 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413dd9b-782c-486c-9b9e-e13cfb6e050d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:11.880447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413dd9b-782c-486c-9b9e-e13cfb6e050d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:11.880744 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413dd9b-782c-486c-9b9e-e13cfb6e050d None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:11.881452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b413dd9b-782c-486c-9b9e-e13cfb6e050d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:11.881753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b413dd9b-782c-486c-9b9e-e13cfb6e050d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:11.883159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b413dd9b-782c-486c-9b9e-e13cfb6e050d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:11.888522 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae37b4f-efc2-43fe-ac98-ca3933bc7ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:11.889157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae37b4f-efc2-43fe-ac98-ca3933bc7ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:11.889441 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae37b4f-efc2-43fe-ac98-ca3933bc7ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:11.889711 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae37b4f-efc2-43fe-ac98-ca3933bc7ea0 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:11.890459 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae37b4f-efc2-43fe-ac98-ca3933bc7ea0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:11.890764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae37b4f-efc2-43fe-ac98-ca3933bc7ea0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:11.892173 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae37b4f-efc2-43fe-ac98-ca3933bc7ea0 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:11.909008 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:11.921407 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 193/385] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Oct 20 05:52:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:11.922696 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18aa779-3fef-4c74-a30f-0726f8d8a999 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:11.922986 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18aa779-3fef-4c74-a30f-0726f8d8a999 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:11.923259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18aa779-3fef-4c74-a30f-0726f8d8a999 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:11.925065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 194/386] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:11.927419 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18aa779-3fef-4c74-a30f-0726f8d8a999 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:11.930111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 193/387] 10.222.0.20 () {56 vars in 1137 bytes} [Thu Oct 20 05:52:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:11.930946 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae37b4f-efc2-43fe-ac98-ca3933bc7ea0 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:12.028784 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18aa779-3fef-4c74-a30f-0726f8d8a999 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:12.029342 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae37b4f-efc2-43fe-ac98-ca3933bc7ea0 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:12.030800 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:12.062842 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56786e6-d21a-4c22-afe1-925126f88921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:12.063390 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56786e6-d21a-4c22-afe1-925126f88921 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:12.063669 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56786e6-d21a-4c22-afe1-925126f88921 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:12.063940 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56786e6-d21a-4c22-afe1-925126f88921 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:12.065011 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e359b33-b18a-4e4a-984a-693cdc68d027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:12.066117 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e359b33-b18a-4e4a-984a-693cdc68d027 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:12.066387 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e359b33-b18a-4e4a-984a-693cdc68d027 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:12.066654 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e359b33-b18a-4e4a-984a-693cdc68d027 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:12.067299 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e359b33-b18a-4e4a-984a-693cdc68d027 None admin] RBAC: Adding query filter params (name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:12.067593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e359b33-b18a-4e4a-984a-693cdc68d027 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:12.068985 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e359b33-b18a-4e4a-984a-693cdc68d027 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:12.072063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56786e6-d21a-4c22-afe1-925126f88921 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:12.073517 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56786e6-d21a-4c22-afe1-925126f88921 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:12.076794 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 194/388] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Oct 20 05:52:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:12.077599 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e359b33-b18a-4e4a-984a-693cdc68d027 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:12.086305 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-c56786e6-d21a-4c22-afe1-925126f88921 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 20 05:52:12.086305 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:12.086305 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:12.086305 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:12.086305 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:12.086305 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:12.086305 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:12.086305 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:12.086305 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:12.086305 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:12.086305 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:12.086305 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:12.086305 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:52:12.086305 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:52:12.086305 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:52:12.086305 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:52:12.087200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:12.087200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:12.087200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:52:12.087200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:12.087200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:52:12.087200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:12.087200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:12.087200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:12.087200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:12.087200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:12.087200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:12.087200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:12.087200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:12.087200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:12.087200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:12.087200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:12.087200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:12.087200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:12.088094 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:12.088094 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:12.088094 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:52:12.088094 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:52:12.088094 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:52:12.088094 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:52:12.088094 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:52:12.088094 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:52:12.088094 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 20 05:52:12.088094 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:12.088094 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 195/389] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:52:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:12.088661 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d5566-c96f-432d-b44a-dfc19b6770f0 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:12.088890 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d5566-c96f-432d-b44a-dfc19b6770f0 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:12.089071 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d5566-c96f-432d-b44a-dfc19b6770f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:12.106573 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-507d5566-c96f-432d-b44a-dfc19b6770f0 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:12.107752 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-507d5566-c96f-432d-b44a-dfc19b6770f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:12.163540 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:12.178368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 196/390] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:52:12.179179 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d5566-c96f-432d-b44a-dfc19b6770f0 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:12.182414 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e359b33-b18a-4e4a-984a-693cdc68d027 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:12.215791 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0db6e-a6ff-489e-a76b-0ffe008e7cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:12.216662 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0db6e-a6ff-489e-a76b-0ffe008e7cac None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:12.216662 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0db6e-a6ff-489e-a76b-0ffe008e7cac None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:12.216942 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0db6e-a6ff-489e-a76b-0ffe008e7cac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:12.228185 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0db6e-a6ff-489e-a76b-0ffe008e7cac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:12.229649 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0db6e-a6ff-489e-a76b-0ffe008e7cac None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:12.244022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-dbb0db6e-a6ff-489e-a76b-0ffe008e7cac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:12.244022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:12.244022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:12.244022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:12.244022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:12.244022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:12.244022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:12.244022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:12.244022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:12.244022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:12.244022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:12.244022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:12.244022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:12.244022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:12.244022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:12.244022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:12.244972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:12.244972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:12.244972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:12.244972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:12.244972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:12.244972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:12.244972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:12.244972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:12.244972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:12.244972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:12.244972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:12.244972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:12.244972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:12.244972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:12.244972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:12.244972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:12.244972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:12.245820 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:12.245820 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:12.245820 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:12.245820 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 195/391] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:12.246054 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0db6e-a6ff-489e-a76b-0ffe008e7cac None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:12.266287 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:12.309821 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d5566-c96f-432d-b44a-dfc19b6770f0 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:12.351885 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd37249-5930-49ac-91de-fa51bc0f67e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:12.352437 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd37249-5930-49ac-91de-fa51bc0f67e3 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:12.352726 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd37249-5930-49ac-91de-fa51bc0f67e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:12.353071 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd37249-5930-49ac-91de-fa51bc0f67e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:12.354423 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0db6e-a6ff-489e-a76b-0ffe008e7cac None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:12.364173 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd37249-5930-49ac-91de-fa51bc0f67e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:12.365660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd37249-5930-49ac-91de-fa51bc0f67e3 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:12.381877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-7fd37249-5930-49ac-91de-fa51bc0f67e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:12.381877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:12.381877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:12.381877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:12.381877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:12.381877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:12.381877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:12.381877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:12.381877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:12.381877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:12.381877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:12.381877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:12.381877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:12.381877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:12.381877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:12.381877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:12.383004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:12.383004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:12.383004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:12.383004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:12.383004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:12.383004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:12.383004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:12.383004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:12.383004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:12.383004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:12.383004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:12.383004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:12.383004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:12.383004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:12.383004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:12.383004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:12.383004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:12.383832 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:12.383832 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:12.383832 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:12.383832 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 197/392] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:12.383832 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd37249-5930-49ac-91de-fa51bc0f67e3 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:12.394377 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c94712-6c38-4bb9-ad30-718c442f2d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:12.395232 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c94712-6c38-4bb9-ad30-718c442f2d76 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:12.395644 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c94712-6c38-4bb9-ad30-718c442f2d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:12.396071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c94712-6c38-4bb9-ad30-718c442f2d76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:12.406799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c94712-6c38-4bb9-ad30-718c442f2d76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:12.408406 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c94712-6c38-4bb9-ad30-718c442f2d76 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:12.423433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-f6c94712-6c38-4bb9-ad30-718c442f2d76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:12.423433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:12.423433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:12.423433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:12.423433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:12.423433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:12.423433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:12.423433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:12.423433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:12.423433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:12.423433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:12.423433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:12.423433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:12.423433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:12.423433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:12.423433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:12.424352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:12.424352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:12.424352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:12.424352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:12.424352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:12.424352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:12.424352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:12.424352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:12.424352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:12.424352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:12.424352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:12.424352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:12.424352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:12.424352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:12.424352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:12.424352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:12.424352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:12.425235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:12.425235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:12.425235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:12.425235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 196/393] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:12.425459 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c94712-6c38-4bb9-ad30-718c442f2d76 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:12.435168 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:12.494749 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:12.498005 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd37249-5930-49ac-91de-fa51bc0f67e3 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:12.529993 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c94712-6c38-4bb9-ad30-718c442f2d76 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:12.531838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679baa1b-d0ad-496c-b24e-7acb8c4d4665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:12.532443 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679baa1b-d0ad-496c-b24e-7acb8c4d4665 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:12.532742 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679baa1b-d0ad-496c-b24e-7acb8c4d4665 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:12.533015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679baa1b-d0ad-496c-b24e-7acb8c4d4665 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:12.533725 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679baa1b-d0ad-496c-b24e-7acb8c4d4665 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:12.534078 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679baa1b-d0ad-496c-b24e-7acb8c4d4665 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:12.535651 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679baa1b-d0ad-496c-b24e-7acb8c4d4665 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:12.552780 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 198/394] 10.222.0.20 () {56 vars in 1137 bytes} [Thu Oct 20 05:52:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:12.553571 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679baa1b-d0ad-496c-b24e-7acb8c4d4665 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:12.562782 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893f458-7515-4d49-bbe6-596495243aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:12.563376 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893f458-7515-4d49-bbe6-596495243aef None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:12.563655 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893f458-7515-4d49-bbe6-596495243aef None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:12.563930 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893f458-7515-4d49-bbe6-596495243aef None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:12.564553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893f458-7515-4d49-bbe6-596495243aef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:12.564882 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893f458-7515-4d49-bbe6-596495243aef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:12.566335 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893f458-7515-4d49-bbe6-596495243aef None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:12.577717 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 197/395] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:12.578479 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893f458-7515-4d49-bbe6-596495243aef None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:12.647875 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:12.651819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679baa1b-d0ad-496c-b24e-7acb8c4d4665 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:12.678150 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893f458-7515-4d49-bbe6-596495243aef None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:12.685292 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f2cad-ad29-4841-aac2-1c3637ba282e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:12.685900 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f2cad-ad29-4841-aac2-1c3637ba282e None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:12.686179 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f2cad-ad29-4841-aac2-1c3637ba282e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:12.686452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f2cad-ad29-4841-aac2-1c3637ba282e None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:12.687083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f2cad-ad29-4841-aac2-1c3637ba282e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:12.687383 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f2cad-ad29-4841-aac2-1c3637ba282e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:12.688771 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f2cad-ad29-4841-aac2-1c3637ba282e None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:12.699680 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 199/396] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:12.700461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f2cad-ad29-4841-aac2-1c3637ba282e None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:12.703308 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:12.712294 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b67f1-8c1c-4d7d-a4fd-87c38855b7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:12.712928 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b67f1-8c1c-4d7d-a4fd-87c38855b7cf None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:12.713212 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b67f1-8c1c-4d7d-a4fd-87c38855b7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:12.713482 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b67f1-8c1c-4d7d-a4fd-87c38855b7cf None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:12.714116 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b67f1-8c1c-4d7d-a4fd-87c38855b7cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:12.714416 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b67f1-8c1c-4d7d-a4fd-87c38855b7cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:12.715790 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b67f1-8c1c-4d7d-a4fd-87c38855b7cf None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:12.727732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 198/397] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:12.728504 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b67f1-8c1c-4d7d-a4fd-87c38855b7cf None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:12.806414 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f2cad-ad29-4841-aac2-1c3637ba282e None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:12.824495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b67f1-8c1c-4d7d-a4fd-87c38855b7cf None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:12.841087 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3a95b-918b-4312-bcfe-fe1dac0a506b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:12.841719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3a95b-918b-4312-bcfe-fe1dac0a506b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:12.842002 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3a95b-918b-4312-bcfe-fe1dac0a506b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:12.842273 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3a95b-918b-4312-bcfe-fe1dac0a506b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:12.853105 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c3a95b-918b-4312-bcfe-fe1dac0a506b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:12.854554 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c3a95b-918b-4312-bcfe-fe1dac0a506b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:12.857276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbfbd8b-11b6-41f5-a15c-f5c9f18c44ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:12.857881 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbfbd8b-11b6-41f5-a15c-f5c9f18c44ac None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:12.858155 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbfbd8b-11b6-41f5-a15c-f5c9f18c44ac None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:12.858420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbfbd8b-11b6-41f5-a15c-f5c9f18c44ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:12.868197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbfbd8b-11b6-41f5-a15c-f5c9f18c44ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:12.868496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-e3c3a95b-918b-4312-bcfe-fe1dac0a506b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:12.868496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:12.868496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:12.868496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:12.868496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:12.868496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:12.868496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:12.868496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:12.868496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:12.868496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:12.868496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:12.868496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:12.868496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:12.868496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:12.868496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:12.868496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:12.869435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:12.869435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:12.869435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:12.869435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:12.869435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:12.869435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:12.869435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:12.869435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:12.869435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:12.869435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:12.869435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:12.869435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:12.869435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:12.869435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:12.869435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:12.869435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:12.869435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:12.870268 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:12.870268 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:12.870268 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:12.870268 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 200/398] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:12.870494 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbfbd8b-11b6-41f5-a15c-f5c9f18c44ac None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:12.870562 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3a95b-918b-4312-bcfe-fe1dac0a506b None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:12.882837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:12.889258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-9bbfbd8b-11b6-41f5-a15c-f5c9f18c44ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:12.889258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:12.889258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:12.889258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:12.889258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:12.889258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:12.889258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:12.889258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:12.889258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:12.889258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:12.889258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:12.889258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:12.889258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:12.889258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:12.889258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:12.889258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:12.890190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:12.890190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:12.890190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:12.890190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:12.890190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:12.890190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:12.890190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:12.890190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:12.890190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:12.890190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:12.890190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:12.890190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:12.890190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:12.890190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:12.890190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:12.890190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:12.890190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:12.891082 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:12.891082 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:12.891082 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:12.891082 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 199/399] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:12.891082 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbfbd8b-11b6-41f5-a15c-f5c9f18c44ac None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:12.932192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:12.975993 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3a95b-918b-4312-bcfe-fe1dac0a506b None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:12.989582 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbfbd8b-11b6-41f5-a15c-f5c9f18c44ac None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:13.009377 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752015e-0122-4cd8-9f4e-e5f482c14fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:13.010252 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752015e-0122-4cd8-9f4e-e5f482c14fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:13.010522 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752015e-0122-4cd8-9f4e-e5f482c14fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:13.010787 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752015e-0122-4cd8-9f4e-e5f482c14fd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:13.017267 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752015e-0122-4cd8-9f4e-e5f482c14fd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:13.018758 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752015e-0122-4cd8-9f4e-e5f482c14fd8 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:13.023199 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e442a28-56bb-43a0-9b51-a358b054d198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:13.023815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e442a28-56bb-43a0-9b51-a358b054d198 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:13.024098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e442a28-56bb-43a0-9b51-a358b054d198 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:13.024361 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e442a28-56bb-43a0-9b51-a358b054d198 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:13.030092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-2752015e-0122-4cd8-9f4e-e5f482c14fd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:13.030092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:13.030092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:13.030092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:13.030092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:13.030092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:13.030092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:13.030092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:13.030092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:13.030092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:13.030092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:13.030092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:13.030092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:52:13.030092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:52:13.030092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:52:13.031065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:52:13.031065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:13.031065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:13.031065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:13.031065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:13.031065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:13.031065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:13.031065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:13.031065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:13.031065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:13.031065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:13.031065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:13.031065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:13.031065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:13.031065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:13.031065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:13.031065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:13.032013 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:52:13.032013 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:52:13.032013 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:52:13.032013 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:52:13.032013 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:52:13.032013 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:52:13.032013 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:13.032013 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:13.032013 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 201/400] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:13.032013 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752015e-0122-4cd8-9f4e-e5f482c14fd8 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:13.033287 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e442a28-56bb-43a0-9b51-a358b054d198 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:13.035728 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e442a28-56bb-43a0-9b51-a358b054d198 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:13.047682 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-3e442a28-56bb-43a0-9b51-a358b054d198 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:13.047682 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:13.047682 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:13.047682 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:13.047682 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:13.047682 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:13.047682 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:13.047682 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:13.047682 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:13.047682 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:13.047682 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:13.047682 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:13.047682 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:52:13.047682 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:52:13.047682 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:52:13.048535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:52:13.048535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:13.048535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:13.048535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:13.048535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:13.048535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:13.048535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:13.048535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:13.048535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:13.048535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:13.048535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:13.048535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:13.048535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:13.048535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:13.048535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:13.048535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:13.048535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:13.050147 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:52:13.050147 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:52:13.050147 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:52:13.050147 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:52:13.050147 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:52:13.050147 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:52:13.050147 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:13.050147 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:13.050147 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 200/401] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:13.050979 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e442a28-56bb-43a0-9b51-a358b054d198 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:13.106593 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:13.137605 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752015e-0122-4cd8-9f4e-e5f482c14fd8 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:13.146907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:13.149835 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e442a28-56bb-43a0-9b51-a358b054d198 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:13.171391 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f31c44-00c0-4811-9a16-e7ca045e79c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:13.172015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f31c44-00c0-4811-9a16-e7ca045e79c4 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:13.172289 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f31c44-00c0-4811-9a16-e7ca045e79c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:13.172561 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f31c44-00c0-4811-9a16-e7ca045e79c4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:13.180746 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f31c44-00c0-4811-9a16-e7ca045e79c4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:13.182188 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f31c44-00c0-4811-9a16-e7ca045e79c4 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:13.183792 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2170b8-a78f-4299-bf86-6c1ba351c83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:13.184331 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2170b8-a78f-4299-bf86-6c1ba351c83c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:13.184612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2170b8-a78f-4299-bf86-6c1ba351c83c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:13.184911 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2170b8-a78f-4299-bf86-6c1ba351c83c None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:13.185538 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2170b8-a78f-4299-bf86-6c1ba351c83c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:13.185840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2170b8-a78f-4299-bf86-6c1ba351c83c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:13.187185 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2170b8-a78f-4299-bf86-6c1ba351c83c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:13.193205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-82f31c44-00c0-4811-9a16-e7ca045e79c4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 20 05:52:13.193205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:13.193205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:13.193205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:13.193205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:13.193205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:13.193205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:13.193205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:13.193205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:13.193205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:13.193205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:13.193205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:13.193205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:52:13.193205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:52:13.193205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:52:13.193205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:52:13.194240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:13.194240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:13.194240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:52:13.194240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:13.194240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:52:13.194240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:13.194240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:13.194240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:13.194240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:13.194240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:13.194240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:13.194240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:13.194240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:13.194240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:13.194240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:13.194240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:13.194240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:13.194240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:13.199777 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:13.199777 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:13.199777 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:52:13.199777 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:52:13.199777 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:52:13.199777 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:52:13.199777 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:52:13.199777 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:52:13.199777 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 20 05:52:13.199777 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:13.199777 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 202/402] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Oct 20 05:52:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:13.199777 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f31c44-00c0-4811-9a16-e7ca045e79c4 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:13.200378 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 201/403] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:13.200378 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2170b8-a78f-4299-bf86-6c1ba351c83c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:13.295380 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2170b8-a78f-4299-bf86-6c1ba351c83c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:13.298672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f31c44-00c0-4811-9a16-e7ca045e79c4 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:13.328950 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:13.329099 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f656927-1c4c-4218-981f-777a6c323835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:13.329666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f656927-1c4c-4218-981f-777a6c323835 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:13.329938 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f656927-1c4c-4218-981f-777a6c323835 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:13.330204 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f656927-1c4c-4218-981f-777a6c323835 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:13.330949 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f656927-1c4c-4218-981f-777a6c323835 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:13.331278 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f656927-1c4c-4218-981f-777a6c323835 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:13.332803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f656927-1c4c-4218-981f-777a6c323835 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:13.335088 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c83c2a-37d1-4f70-a605-10b4d943ed05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:13.335692 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c83c2a-37d1-4f70-a605-10b4d943ed05 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:13.335972 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c83c2a-37d1-4f70-a605-10b4d943ed05 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:13.336245 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c83c2a-37d1-4f70-a605-10b4d943ed05 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:13.336917 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c83c2a-37d1-4f70-a605-10b4d943ed05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:13.337221 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c83c2a-37d1-4f70-a605-10b4d943ed05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:13.338575 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c83c2a-37d1-4f70-a605-10b4d943ed05 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:13.349699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 202/404] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Oct 20 05:52:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:13.354873 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 203/405] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:13.354988 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f656927-1c4c-4218-981f-777a6c323835 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:13.355074 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c83c2a-37d1-4f70-a605-10b4d943ed05 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:13.376425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:13.450826 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c83c2a-37d1-4f70-a605-10b4d943ed05 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:13.455037 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f656927-1c4c-4218-981f-777a6c323835 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:13.484492 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c23496-f41b-47df-a959-eb3dd0571633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:13.485119 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c23496-f41b-47df-a959-eb3dd0571633 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:13.485395 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c23496-f41b-47df-a959-eb3dd0571633 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:13.485662 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c23496-f41b-47df-a959-eb3dd0571633 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:13.486287 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c23496-f41b-47df-a959-eb3dd0571633 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:13.486948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3598cde-9ffe-4fe4-82a3-3e91481bedce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:13.488056 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3598cde-9ffe-4fe4-82a3-3e91481bedce None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:13.488312 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c23496-f41b-47df-a959-eb3dd0571633 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:13.488401 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3598cde-9ffe-4fe4-82a3-3e91481bedce None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:13.488595 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3598cde-9ffe-4fe4-82a3-3e91481bedce None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:13.489307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3598cde-9ffe-4fe4-82a3-3e91481bedce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:13.489613 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3598cde-9ffe-4fe4-82a3-3e91481bedce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:13.491037 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3598cde-9ffe-4fe4-82a3-3e91481bedce None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:13.503414 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 203/406] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Oct 20 05:52:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:13.504276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3598cde-9ffe-4fe4-82a3-3e91481bedce None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:13.513941 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 204/407] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Oct 20 05:52:13 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:52:13.515126 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c23496-f41b-47df-a959-eb3dd0571633 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:13.571236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:13.599881 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:13.623684 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3598cde-9ffe-4fe4-82a3-3e91481bedce None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:13.634399 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c23496-f41b-47df-a959-eb3dd0571633 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:13.662956 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302de4cb-7977-4568-a4f5-a7594f0bdcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:13.663571 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302de4cb-7977-4568-a4f5-a7594f0bdcc8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:13.663845 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302de4cb-7977-4568-a4f5-a7594f0bdcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:13.664120 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302de4cb-7977-4568-a4f5-a7594f0bdcc8 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:13.664898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302de4cb-7977-4568-a4f5-a7594f0bdcc8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:13.665203 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302de4cb-7977-4568-a4f5-a7594f0bdcc8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:13.666649 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302de4cb-7977-4568-a4f5-a7594f0bdcc8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:13.673663 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01396ad-7944-45b7-9e98-6ed8a378296f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:13.674625 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01396ad-7944-45b7-9e98-6ed8a378296f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:13.674941 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01396ad-7944-45b7-9e98-6ed8a378296f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:13.675241 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01396ad-7944-45b7-9e98-6ed8a378296f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:13.684811 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01396ad-7944-45b7-9e98-6ed8a378296f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:13.686790 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01396ad-7944-45b7-9e98-6ed8a378296f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:13.687207 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 204/408] 10.222.0.20 () {56 vars in 1139 bytes} [Thu Oct 20 05:52:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:13.688003 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302de4cb-7977-4568-a4f5-a7594f0bdcc8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:13.699436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-b01396ad-7944-45b7-9e98-6ed8a378296f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:13.699436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:13.699436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:13.699436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:13.699436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:13.699436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:13.699436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:13.699436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:13.699436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:13.699436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:13.699436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:13.699436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:13.699436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:52:13.699436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:52:13.699436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:52:13.700293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:52:13.700293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:13.700293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:13.700293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:13.700293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:13.700293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:13.700293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:13.700293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:13.700293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:13.700293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:13.700293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:13.700293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:13.700293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:13.700293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:13.700293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:13.700293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:13.700293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:13.701638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:52:13.701638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:52:13.701638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:52:13.701638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:52:13.701638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:52:13.701638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:52:13.701638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:13.701638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:13.701638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 205/409] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:13.701638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01396ad-7944-45b7-9e98-6ed8a378296f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:13.791472 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302de4cb-7977-4568-a4f5-a7594f0bdcc8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:13.810840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01396ad-7944-45b7-9e98-6ed8a378296f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:13.818347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:13.826802 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e83502-d3d7-4ecd-bf4c-a6391c27cea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:13.827841 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e83502-d3d7-4ecd-bf4c-a6391c27cea8 None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:13.828275 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e83502-d3d7-4ecd-bf4c-a6391c27cea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:13.828556 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e83502-d3d7-4ecd-bf4c-a6391c27cea8 None admin] PATH_INFO: `/v3/projects/0ee65b9a8229495bbe97318f060d6342/users/ae2c1c886cd94d9f90082f66e7e8c9f4/roles/afd859d2deed4d51aa3f6326920441c8` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:13.837300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:13.845799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e83502-d3d7-4ecd-bf4c-a6391c27cea8 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ee65b9a8229495bbe97318f060d6342, user_id=ae2c1c886cd94d9f90082f66e7e8c9f4, role_id=afd859d2deed4d51aa3f6326920441c8)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:13.847408 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8efa22-8cf4-4153-a475-62a4d35a3643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:13.847902 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e83502-d3d7-4ecd-bf4c-a6391c27cea8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:13.848156 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8efa22-8cf4-4153-a475-62a4d35a3643 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:13.848436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8efa22-8cf4-4153-a475-62a4d35a3643 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:13.848733 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8efa22-8cf4-4153-a475-62a4d35a3643 None admin] PATH_INFO: `/v3/projects/0ee65b9a8229495bbe97318f060d6342/users/cfb00a74a99e47c1ac938f07d26a4bc8/roles/afd859d2deed4d51aa3f6326920441c8` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:13.862352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 205/410] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:52:13 2022] PUT /identity/v3/projects/0ee65b9a8229495bbe97318f060d6342/users/ae2c1c886cd94d9f90082f66e7e8c9f4/roles/afd859d2deed4d51aa3f6326920441c8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:52:13.863670 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e83502-d3d7-4ecd-bf4c-a6391c27cea8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:13.865239 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8efa22-8cf4-4153-a475-62a4d35a3643 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ee65b9a8229495bbe97318f060d6342, user_id=cfb00a74a99e47c1ac938f07d26a4bc8, role_id=afd859d2deed4d51aa3f6326920441c8)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:13.866736 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8efa22-8cf4-4153-a475-62a4d35a3643 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:13.900971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 206/411] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:52:13 2022] PUT /identity/v3/projects/0ee65b9a8229495bbe97318f060d6342/users/cfb00a74a99e47c1ac938f07d26a4bc8/roles/afd859d2deed4d51aa3f6326920441c8 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:52:13.903037 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8efa22-8cf4-4153-a475-62a4d35a3643 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:14.041356 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e83502-d3d7-4ecd-bf4c-a6391c27cea8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:14.108839 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e86efc-73b3-466e-aa88-d2108cb0e2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:14.109491 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e86efc-73b3-466e-aa88-d2108cb0e2c4 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:14.109768 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e86efc-73b3-466e-aa88-d2108cb0e2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:14.110035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e86efc-73b3-466e-aa88-d2108cb0e2c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:14.125362 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8efa22-8cf4-4153-a475-62a4d35a3643 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:14.130269 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e86efc-73b3-466e-aa88-d2108cb0e2c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:14.131745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e86efc-73b3-466e-aa88-d2108cb0e2c4 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:14.150900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-21e86efc-73b3-466e-aa88-d2108cb0e2c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:14.150900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:14.150900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:14.150900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:14.150900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:14.150900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:14.150900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:14.150900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:14.150900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:14.150900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:14.150900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:14.150900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:14.150900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:14.150900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:14.150900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:14.150900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:14.152260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:14.152260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:14.152260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:14.152260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:14.152260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:14.152260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:14.152260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:14.152260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:14.152260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:14.152260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:14.152260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:14.152260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:14.152260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:14.152260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:14.152260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:14.152260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:14.152260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:14.153135 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:14.153135 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:14.153135 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:14.153135 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 206/412] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:14.160329 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e86efc-73b3-466e-aa88-d2108cb0e2c4 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:14.171792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ae95c-8744-4595-b1ce-31ce27249eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:14.172415 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ae95c-8744-4595-b1ce-31ce27249eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:14.173184 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ae95c-8744-4595-b1ce-31ce27249eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:14.173469 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ae95c-8744-4595-b1ce-31ce27249eb6 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:14.174163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ae95c-8744-4595-b1ce-31ce27249eb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:14.174855 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ae95c-8744-4595-b1ce-31ce27249eb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:14.176483 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ae95c-8744-4595-b1ce-31ce27249eb6 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:14.200471 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 207/413] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Oct 20 05:52:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:14.206704 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ae95c-8744-4595-b1ce-31ce27249eb6 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:14.231423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:14.237404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:14.293386 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e86efc-73b3-466e-aa88-d2108cb0e2c4 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:14.341470 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640dd6bd-d279-4e43-80f8-1e78016c9d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:14.342420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640dd6bd-d279-4e43-80f8-1e78016c9d0b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:14.342708 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640dd6bd-d279-4e43-80f8-1e78016c9d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:14.342981 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640dd6bd-d279-4e43-80f8-1e78016c9d0b None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:14.343908 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640dd6bd-d279-4e43-80f8-1e78016c9d0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:14.344221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640dd6bd-d279-4e43-80f8-1e78016c9d0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:14.345669 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640dd6bd-d279-4e43-80f8-1e78016c9d0b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:14.346233 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ae95c-8744-4595-b1ce-31ce27249eb6 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:14.370737 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 207/414] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:14.375696 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640dd6bd-d279-4e43-80f8-1e78016c9d0b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:14.418675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40199d9e-a6ad-48bc-8794-89639684db32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:14.419699 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40199d9e-a6ad-48bc-8794-89639684db32 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:14.420220 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40199d9e-a6ad-48bc-8794-89639684db32 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:14.420707 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40199d9e-a6ad-48bc-8794-89639684db32 None admin] PATH_INFO: `/v3/projects/0ee65b9a8229495bbe97318f060d6342/users/3dcfe5244bd04c00b6dde1d6f795d4ec/roles/afd859d2deed4d51aa3f6326920441c8` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:14.453894 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40199d9e-a6ad-48bc-8794-89639684db32 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ee65b9a8229495bbe97318f060d6342, user_id=3dcfe5244bd04c00b6dde1d6f795d4ec, role_id=afd859d2deed4d51aa3f6326920441c8)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:14.455696 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40199d9e-a6ad-48bc-8794-89639684db32 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:14.478339 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 208/415] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:52:14 2022] PUT /identity/v3/projects/0ee65b9a8229495bbe97318f060d6342/users/3dcfe5244bd04c00b6dde1d6f795d4ec/roles/afd859d2deed4d51aa3f6326920441c8 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:52:14.556630 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:14.559838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640dd6bd-d279-4e43-80f8-1e78016c9d0b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:14.619017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce33c28-1512-4e21-8c3a-f79a3f6d5756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:14.619678 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce33c28-1512-4e21-8c3a-f79a3f6d5756 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:14.619953 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce33c28-1512-4e21-8c3a-f79a3f6d5756 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:14.620231 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce33c28-1512-4e21-8c3a-f79a3f6d5756 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:14.630639 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce33c28-1512-4e21-8c3a-f79a3f6d5756 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:14.632104 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce33c28-1512-4e21-8c3a-f79a3f6d5756 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:14.647502 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-6ce33c28-1512-4e21-8c3a-f79a3f6d5756 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:14.647502 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:14.647502 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:14.647502 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:14.647502 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:14.647502 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:14.647502 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:14.647502 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:14.647502 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:14.647502 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:14.647502 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:14.647502 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:14.647502 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:52:14.647502 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:52:14.647502 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:52:14.648464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:52:14.648464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:14.648464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:14.648464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:14.648464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:14.648464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:14.648464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:14.648464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:14.648464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:14.648464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:14.648464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:14.648464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:14.648464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:14.648464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:14.648464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:14.648464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:14.648464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:14.649345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:52:14.649345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:52:14.649345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:52:14.649345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:52:14.649345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:52:14.649345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:52:14.649345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:14.649345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:14.650263 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 208/416] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:14.654404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40199d9e-a6ad-48bc-8794-89639684db32 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:14.731437 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:14.772430 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40199d9e-a6ad-48bc-8794-89639684db32 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:14.808813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ee5c8-3042-4f16-a448-34c99287ef63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:14.809740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ee5c8-3042-4f16-a448-34c99287ef63 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:14.810240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ee5c8-3042-4f16-a448-34c99287ef63 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:14.810708 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ee5c8-3042-4f16-a448-34c99287ef63 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:14.811619 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944ee5c8-3042-4f16-a448-34c99287ef63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:14.812549 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944ee5c8-3042-4f16-a448-34c99287ef63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:14.814282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944ee5c8-3042-4f16-a448-34c99287ef63 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:14.828808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 209/417] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Oct 20 05:52:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:14.837374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce33c28-1512-4e21-8c3a-f79a3f6d5756 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:14.995999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce33c28-1512-4e21-8c3a-f79a3f6d5756 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:15.050058 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9b789-6236-4043-a46f-858d977102d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:15.050728 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9b789-6236-4043-a46f-858d977102d9 None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:15.051008 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9b789-6236-4043-a46f-858d977102d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:15.051279 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9b789-6236-4043-a46f-858d977102d9 None admin] PATH_INFO: `/v3/projects/0ee65b9a8229495bbe97318f060d6342/users/9072ce66e7174837aeb8c7898026fcce/roles/afd859d2deed4d51aa3f6326920441c8` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:15.070729 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b9b789-6236-4043-a46f-858d977102d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ee65b9a8229495bbe97318f060d6342, user_id=9072ce66e7174837aeb8c7898026fcce, role_id=afd859d2deed4d51aa3f6326920441c8)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:15.072118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b9b789-6236-4043-a46f-858d977102d9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:15.094205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 209/418] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:52:14 2022] PUT /identity/v3/projects/0ee65b9a8229495bbe97318f060d6342/users/9072ce66e7174837aeb8c7898026fcce/roles/afd859d2deed4d51aa3f6326920441c8 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:52:16.055604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ba97c-0a76-43bf-a7cf-36f66f57c489 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:16.056640 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ba97c-0a76-43bf-a7cf-36f66f57c489 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:16.057165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ba97c-0a76-43bf-a7cf-36f66f57c489 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:16.058984 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 210/419] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:16.064058 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7d5fe-0008-43a2-854d-af5798e3288c None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:16.064485 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7d5fe-0008-43a2-854d-af5798e3288c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:16.064582 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7d5fe-0008-43a2-854d-af5798e3288c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:16.095444 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-ece7d5fe-0008-43a2-854d-af5798e3288c None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:16.097183 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ece7d5fe-0008-43a2-854d-af5798e3288c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:16.105802 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:16.184982 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 210/420] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:52:16.414910 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962c461-da62-46b6-bd75-04b74e86b996 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:16.415278 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962c461-da62-46b6-bd75-04b74e86b996 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:16.415443 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962c461-da62-46b6-bd75-04b74e86b996 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:16.417069 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 211/421] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:16.422332 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705395a-907b-45d4-a9d4-610968f32c63 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:16.422860 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705395a-907b-45d4-a9d4-610968f32c63 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:16.423334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705395a-907b-45d4-a9d4-610968f32c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:16.448575 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-5705395a-907b-45d4-a9d4-610968f32c63 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:16.450114 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5705395a-907b-45d4-a9d4-610968f32c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:16.507033 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962c461-da62-46b6-bd75-04b74e86b996 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:16.560121 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 211/422] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:52:16.673126 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962c461-da62-46b6-bd75-04b74e86b996 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:16.707911 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:16.723623 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9401e72-5f14-4ee8-9d19-ffd3e3217c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:16.724287 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9401e72-5f14-4ee8-9d19-ffd3e3217c1b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:16.724570 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9401e72-5f14-4ee8-9d19-ffd3e3217c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:16.724879 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9401e72-5f14-4ee8-9d19-ffd3e3217c1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:16.740944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9401e72-5f14-4ee8-9d19-ffd3e3217c1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:52:16.745886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-b9401e72-5f14-4ee8-9d19-ffd3e3217c1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:52:16.745886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:16.745886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:16.745886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:16.745886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:16.745886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:16.745886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:16.745886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:16.745886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:16.745886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:16.745886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:16.745886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:16.745886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:52:16.745886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:52:16.745886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:52:16.745886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:52:16.746831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:52:16.746831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:52:16.746831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:52:16.746831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:52:16.746831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:16.746831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:16.746831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:16.746831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:16.746831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:16.746831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:16.746831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:16.746831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:16.746831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:16.746831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:16.746831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:16.746831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:16.747604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:16.747604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:16.747604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:16.747604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:16.747604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:52:16.747604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:52:16.747604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:52:16.747604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:52:16.747604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:52:16.747604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:52:16.747604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:52:16.747604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:16.747604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 212/423] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Oct 20 05:52:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:16.755063 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705395a-907b-45d4-a9d4-610968f32c63 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:16.812227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f480c-b527-44cf-b32a-bf2472c0058c None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:16.812542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f480c-b527-44cf-b32a-bf2472c0058c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:16.812811 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f480c-b527-44cf-b32a-bf2472c0058c None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:16.814409 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 213/424] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:16.825766 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a14198-72fc-4ac7-8cf8-7dcc05887e3c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:16.826052 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a14198-72fc-4ac7-8cf8-7dcc05887e3c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:16.826326 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a14198-72fc-4ac7-8cf8-7dcc05887e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:16.853249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-20a14198-72fc-4ac7-8cf8-7dcc05887e3c None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:16.854531 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-20a14198-72fc-4ac7-8cf8-7dcc05887e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:16.867369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:16.907376 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705395a-907b-45d4-a9d4-610968f32c63 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:16.931709 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 214/425] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 05:52:16.932868 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a14198-72fc-4ac7-8cf8-7dcc05887e3c None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:16.953311 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85695832-1bea-4e4e-8253-6ee9611420d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:16.953929 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85695832-1bea-4e4e-8253-6ee9611420d9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:16.954210 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85695832-1bea-4e4e-8253-6ee9611420d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:16.954485 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85695832-1bea-4e4e-8253-6ee9611420d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:16.955595 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85695832-1bea-4e4e-8253-6ee9611420d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:16.956449 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85695832-1bea-4e4e-8253-6ee9611420d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:16.958436 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85695832-1bea-4e4e-8253-6ee9611420d9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:16.970462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 212/426] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Oct 20 05:52:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:16.981698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85695832-1bea-4e4e-8253-6ee9611420d9 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:17.041075 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a14198-72fc-4ac7-8cf8-7dcc05887e3c None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:17.058806 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:17.083623 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9745691-4b47-4827-89f0-2688b38b58a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:17.084255 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9745691-4b47-4827-89f0-2688b38b58a7 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:17.084529 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9745691-4b47-4827-89f0-2688b38b58a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:17.084827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9745691-4b47-4827-89f0-2688b38b58a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:17.110399 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9745691-4b47-4827-89f0-2688b38b58a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:52:17.117808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-d9745691-4b47-4827-89f0-2688b38b58a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:52:17.117808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:17.117808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:17.117808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:17.117808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:17.117808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:17.117808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:17.117808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:17.117808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:17.117808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:17.117808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:17.117808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:17.117808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:52:17.117808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:52:17.117808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:52:17.117808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:52:17.118655 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:52:17.118655 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:52:17.118655 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:52:17.118655 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:52:17.118655 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:17.118655 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:17.118655 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:17.118655 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:17.118655 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:17.118655 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:17.118655 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:17.118655 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:17.118655 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:17.118655 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:17.118655 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:17.118655 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:17.119454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:17.119454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:17.119454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:17.119454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:17.119454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:52:17.119454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:52:17.119454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:52:17.119454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:52:17.119454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:52:17.119454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:52:17.119454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:52:17.119454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:17.120050 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 215/427] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Oct 20 05:52:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:17.134335 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9745691-4b47-4827-89f0-2688b38b58a7 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:17.155289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85695832-1bea-4e4e-8253-6ee9611420d9 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:17.220276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:17.228637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc016f98-0088-4515-adf0-cc2e96a6963f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:17.229322 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc016f98-0088-4515-adf0-cc2e96a6963f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:17.229603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc016f98-0088-4515-adf0-cc2e96a6963f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:17.229876 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc016f98-0088-4515-adf0-cc2e96a6963f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:17.242809 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc016f98-0088-4515-adf0-cc2e96a6963f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:17.244357 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc016f98-0088-4515-adf0-cc2e96a6963f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:17.259541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-fc016f98-0088-4515-adf0-cc2e96a6963f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:17.259541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:17.259541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:17.259541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:17.259541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:17.259541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:17.259541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:17.259541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:17.259541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:17.259541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:17.259541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:17.259541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:17.259541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:17.259541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:17.259541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:17.259541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:17.260941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:17.260941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:17.260941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:17.260941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:17.260941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:17.260941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:17.260941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:17.260941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:17.260941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:17.260941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:17.260941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:17.260941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:17.260941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:17.260941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:17.260941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:17.260941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:17.260941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:17.261980 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:17.261980 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:17.261980 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:17.263751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 213/428] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:17.264900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc016f98-0088-4515-adf0-cc2e96a6963f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:17.283857 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9745691-4b47-4827-89f0-2688b38b58a7 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:17.328059 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7effb8-7d2b-4b33-b396-f34f7e18fb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:17.328662 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7effb8-7d2b-4b33-b396-f34f7e18fb81 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:17.328971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7effb8-7d2b-4b33-b396-f34f7e18fb81 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:17.329238 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7effb8-7d2b-4b33-b396-f34f7e18fb81 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:17.329894 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7effb8-7d2b-4b33-b396-f34f7e18fb81 None admin] RBAC: Adding query filter params (name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:17.330190 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7effb8-7d2b-4b33-b396-f34f7e18fb81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:17.332373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7effb8-7d2b-4b33-b396-f34f7e18fb81 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:17.340348 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 216/429] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Oct 20 05:52:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:17.340348 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7effb8-7d2b-4b33-b396-f34f7e18fb81 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:17.382144 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:17.397092 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc016f98-0088-4515-adf0-cc2e96a6963f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:17.432060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70dc55-7884-4d18-9cf9-075cf877b853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:17.432685 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70dc55-7884-4d18-9cf9-075cf877b853 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:17.432984 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70dc55-7884-4d18-9cf9-075cf877b853 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:17.433250 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70dc55-7884-4d18-9cf9-075cf877b853 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:17.444334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f70dc55-7884-4d18-9cf9-075cf877b853 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:52:17.449222 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7effb8-7d2b-4b33-b396-f34f7e18fb81 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:17.450354 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-3f70dc55-7884-4d18-9cf9-075cf877b853 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:52:17.450354 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:17.450354 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:17.450354 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:17.450354 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:17.450354 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:17.450354 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:17.450354 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:17.450354 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:17.450354 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:17.450354 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:17.450354 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:17.450354 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:52:17.450354 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:52:17.450354 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:52:17.450354 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:52:17.451260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:52:17.451260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:52:17.451260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:52:17.451260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:52:17.451260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:17.451260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:17.451260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:17.451260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:17.451260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:17.451260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:17.451260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:17.451260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:17.451260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:17.451260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:17.451260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:17.451260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:17.452060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:17.452060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:17.452060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:17.452060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:17.452060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:52:17.452060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:52:17.452060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:52:17.452060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:52:17.452060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:52:17.452060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:52:17.452060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:52:17.452060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:17.452060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 214/430] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Oct 20 05:52:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:17.452739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ff792-cc97-43b2-a2ad-a45bc9f53b09 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:17.453401 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ff792-cc97-43b2-a2ad-a45bc9f53b09 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:17.453401 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ff792-cc97-43b2-a2ad-a45bc9f53b09 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:17.456823 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 215/431] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:17.456823 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ff792-cc97-43b2-a2ad-a45bc9f53b09 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:17.496086 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1ecfa-4cdb-4fad-9bea-ccc690de9e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:17.496622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1ecfa-4cdb-4fad-9bea-ccc690de9e8a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:17.496933 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1ecfa-4cdb-4fad-9bea-ccc690de9e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:17.497209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1ecfa-4cdb-4fad-9bea-ccc690de9e8a None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:17.497844 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e1ecfa-4cdb-4fad-9bea-ccc690de9e8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:17.498145 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e1ecfa-4cdb-4fad-9bea-ccc690de9e8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:17.499942 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e1ecfa-4cdb-4fad-9bea-ccc690de9e8a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:17.506643 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:17.519442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 217/432] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:17.520751 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1ecfa-4cdb-4fad-9bea-ccc690de9e8a None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:17.563689 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ff792-cc97-43b2-a2ad-a45bc9f53b09 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:17.598395 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8b45a-b65f-4975-98ed-85520f2de7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:17.599002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8b45a-b65f-4975-98ed-85520f2de7dd None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:17.599279 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8b45a-b65f-4975-98ed-85520f2de7dd None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:17.599552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8b45a-b65f-4975-98ed-85520f2de7dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:17.609629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8b45a-b65f-4975-98ed-85520f2de7dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:17.611063 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8b45a-b65f-4975-98ed-85520f2de7dd None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:17.626541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-f6b8b45a-b65f-4975-98ed-85520f2de7dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:17.626541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:17.626541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:17.626541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:17.626541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:17.626541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:17.626541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:17.626541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:17.626541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:17.626541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:17.626541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:17.626541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:17.626541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:17.626541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:17.626541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:17.626541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:17.628006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:17.628006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:17.628006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:17.628006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:17.628006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:17.628006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:17.628006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:17.628006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:17.628006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:17.628006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:17.628006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:17.628006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:17.628006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:17.628006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:17.628006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:17.628006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:17.628006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:17.634387 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:17.634387 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:17.634387 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:17.634387 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 216/433] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:17.634387 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1eec7-5440-40f4-928b-a1289e0d63d2 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:17.634387 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1eec7-5440-40f4-928b-a1289e0d63d2 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:17.634387 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1eec7-5440-40f4-928b-a1289e0d63d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:17.635174 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:17.638495 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1ecfa-4cdb-4fad-9bea-ccc690de9e8a None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:17.650858 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-dae1eec7-5440-40f4-928b-a1289e0d63d2 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:17.652118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-dae1eec7-5440-40f4-928b-a1289e0d63d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:17.674270 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb52aa-7c97-467a-b23f-3444756c77c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:17.675386 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb52aa-7c97-467a-b23f-3444756c77c7 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:17.675659 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb52aa-7c97-467a-b23f-3444756c77c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:17.675923 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb52aa-7c97-467a-b23f-3444756c77c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:17.676562 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bb52aa-7c97-467a-b23f-3444756c77c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:17.676883 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bb52aa-7c97-467a-b23f-3444756c77c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:17.678240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bb52aa-7c97-467a-b23f-3444756c77c7 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:17.686826 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 218/434] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Oct 20 05:52:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:17.687518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb52aa-7c97-467a-b23f-3444756c77c7 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:17.736733 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 217/435] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:52:17.736733 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1eec7-5440-40f4-928b-a1289e0d63d2 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:17.761368 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:17.813990 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb52aa-7c97-467a-b23f-3444756c77c7 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:17.841666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1eec7-5440-40f4-928b-a1289e0d63d2 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:17.858062 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb54e83-4d36-45e2-98bb-4e6d30fbc508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:17.858909 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb54e83-4d36-45e2-98bb-4e6d30fbc508 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:17.859312 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb54e83-4d36-45e2-98bb-4e6d30fbc508 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:17.859853 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb54e83-4d36-45e2-98bb-4e6d30fbc508 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:17.869233 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb54e83-4d36-45e2-98bb-4e6d30fbc508 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:17.870813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb54e83-4d36-45e2-98bb-4e6d30fbc508 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:17.877504 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da53b9f-5c72-45ab-a65b-76acc836fdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:17.878134 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da53b9f-5c72-45ab-a65b-76acc836fdb9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:17.878407 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da53b9f-5c72-45ab-a65b-76acc836fdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:17.878822 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da53b9f-5c72-45ab-a65b-76acc836fdb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:17.879598 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da53b9f-5c72-45ab-a65b-76acc836fdb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:17.879904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da53b9f-5c72-45ab-a65b-76acc836fdb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:17.881470 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da53b9f-5c72-45ab-a65b-76acc836fdb9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:17.885447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-5fb54e83-4d36-45e2-98bb-4e6d30fbc508 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 20 05:52:17.885447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:17.885447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:17.885447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:17.885447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:17.885447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:17.885447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:17.885447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:17.885447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:17.885447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:17.885447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:17.885447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:17.885447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:52:17.885447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:52:17.885447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:52:17.885447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:52:17.886324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:17.886324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:17.886324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:52:17.886324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:17.886324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:52:17.886324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:17.886324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:17.886324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:17.886324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:17.886324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:17.886324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:17.886324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:17.886324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:17.886324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:17.886324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:17.886324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:17.886324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:17.886324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:17.887213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:17.887213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:17.887213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:52:17.887213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:52:17.887213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:52:17.887213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:52:17.887213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:52:17.887213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:52:17.887213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 20 05:52:17.887213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:17.887213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 219/436] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Oct 20 05:52:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:17.887764 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb54e83-4d36-45e2-98bb-4e6d30fbc508 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:17.895037 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 218/437] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:17.895037 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da53b9f-5c72-45ab-a65b-76acc836fdb9 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:17.921499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:17.989073 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:18.005266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da53b9f-5c72-45ab-a65b-76acc836fdb9 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:18.008476 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb54e83-4d36-45e2-98bb-4e6d30fbc508 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:18.040005 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa58dc-c3bb-4e55-93ab-1d554abdb2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:18.040531 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa58dc-c3bb-4e55-93ab-1d554abdb2ed None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:18.040830 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa58dc-c3bb-4e55-93ab-1d554abdb2ed None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:18.041099 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa58dc-c3bb-4e55-93ab-1d554abdb2ed None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:18.041833 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa58dc-c3bb-4e55-93ab-1d554abdb2ed None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:18.042134 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa58dc-c3bb-4e55-93ab-1d554abdb2ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:18.043565 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa58dc-c3bb-4e55-93ab-1d554abdb2ed None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:18.044172 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb23fb-0020-40de-afd3-96005bb86237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:18.044833 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb23fb-0020-40de-afd3-96005bb86237 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:18.045289 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb23fb-0020-40de-afd3-96005bb86237 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:18.045569 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb23fb-0020-40de-afd3-96005bb86237 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:18.059433 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb23fb-0020-40de-afd3-96005bb86237 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:18.060914 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb23fb-0020-40de-afd3-96005bb86237 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:18.061014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 219/438] 10.222.0.20 () {56 vars in 1135 bytes} [Thu Oct 20 05:52:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:18.061812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa58dc-c3bb-4e55-93ab-1d554abdb2ed None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:18.075464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-accb23fb-0020-40de-afd3-96005bb86237 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:18.075464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:18.075464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:18.075464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:18.075464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:18.075464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:18.075464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:18.075464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:18.075464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:18.075464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:18.075464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:18.075464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:18.075464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:18.075464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:18.075464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:18.075464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:18.076389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:18.076389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:18.076389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:18.076389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:18.076389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:18.076389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:18.076389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:18.076389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:18.076389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:18.076389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:18.076389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:18.076389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:18.076389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:18.076389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:18.076389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:18.076389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:18.076389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:18.077840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:18.077840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:18.077840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:18.077840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 220/439] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:18.077840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d816e5-59fb-47f5-8de8-7505082d6bb8 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:18.078394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d816e5-59fb-47f5-8de8-7505082d6bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:18.078394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d816e5-59fb-47f5-8de8-7505082d6bb8 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:18.079582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 221/440] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:18.080127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d816e5-59fb-47f5-8de8-7505082d6bb8 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:18.152529 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:18.164381 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa58dc-c3bb-4e55-93ab-1d554abdb2ed None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:18.185360 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d816e5-59fb-47f5-8de8-7505082d6bb8 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:18.200094 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a4ad3-392b-4be9-a547-c03405eff8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:18.200748 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a4ad3-392b-4be9-a547-c03405eff8a0 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:18.201046 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a4ad3-392b-4be9-a547-c03405eff8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:18.201303 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a4ad3-392b-4be9-a547-c03405eff8a0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:18.209063 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a4ad3-392b-4be9-a547-c03405eff8a0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:18.210503 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a4ad3-392b-4be9-a547-c03405eff8a0 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:18.217704 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667c4a8-2a07-4714-b9db-8cbb33a0bbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:18.218329 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667c4a8-2a07-4714-b9db-8cbb33a0bbde None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:18.218610 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667c4a8-2a07-4714-b9db-8cbb33a0bbde None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:18.218882 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667c4a8-2a07-4714-b9db-8cbb33a0bbde None admin] PATH_INFO: `/v3/roles/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:18.222767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-e02a4ad3-392b-4be9-a547-c03405eff8a0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 20 05:52:18.222767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:18.222767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:18.222767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:18.222767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:18.222767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:18.222767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:18.222767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:18.222767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:18.222767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:18.222767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:18.222767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:18.222767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:52:18.222767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:52:18.222767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:52:18.222767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:52:18.223629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:18.223629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:18.223629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:52:18.223629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:18.223629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:52:18.223629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:18.223629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:18.223629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:18.223629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:18.223629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:18.223629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:18.223629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:18.223629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:18.223629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:18.223629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:18.223629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:18.223629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:18.223629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:18.224507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:18.224507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:18.224507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:52:18.224507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:52:18.224507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:52:18.224507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:52:18.224507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:52:18.224507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:52:18.224507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 20 05:52:18.224507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:18.224507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 220/441] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Oct 20 05:52:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:18.225164 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a4ad3-392b-4be9-a547-c03405eff8a0 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:18.229559 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:18.232089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4667c4a8-2a07-4714-b9db-8cbb33a0bbde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:52:18.237638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-4667c4a8-2a07-4714-b9db-8cbb33a0bbde None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:52:18.237638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:18.237638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:18.237638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:18.237638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:18.237638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:18.237638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:18.237638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:18.237638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:18.237638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:18.237638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:18.237638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:18.237638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:52:18.237638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:52:18.237638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:52:18.237638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:52:18.238635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:52:18.238635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:52:18.238635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:52:18.238635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:52:18.238635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:18.238635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:18.238635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:18.238635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:18.238635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:18.238635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:18.238635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:18.238635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:18.238635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:18.238635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:18.238635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:18.238635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:18.239646 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:18.239646 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:18.239646 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:18.239646 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:18.239646 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:52:18.239646 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:52:18.239646 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:52:18.239646 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:52:18.239646 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:52:18.239646 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:52:18.239646 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:52:18.239646 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:18.239646 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 222/442] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Oct 20 05:52:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:18.240509 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667c4a8-2a07-4714-b9db-8cbb33a0bbde None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:18.329567 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a4ad3-392b-4be9-a547-c03405eff8a0 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:18.343224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667c4a8-2a07-4714-b9db-8cbb33a0bbde None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:18.362542 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6347b-a92d-47da-a49a-9ff45111c559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:18.363247 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6347b-a92d-47da-a49a-9ff45111c559 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:18.363536 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6347b-a92d-47da-a49a-9ff45111c559 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:18.363789 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6347b-a92d-47da-a49a-9ff45111c559 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:18.371694 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:18.374485 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a6347b-a92d-47da-a49a-9ff45111c559 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:18.375918 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a6347b-a92d-47da-a49a-9ff45111c559 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:18.377484 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d22a4-131a-43f7-b73b-353946e09a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:18.378092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d22a4-131a-43f7-b73b-353946e09a85 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:18.378368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d22a4-131a-43f7-b73b-353946e09a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:18.378643 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d22a4-131a-43f7-b73b-353946e09a85 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:18.379280 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d22a4-131a-43f7-b73b-353946e09a85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:18.379580 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d22a4-131a-43f7-b73b-353946e09a85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:18.380978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d22a4-131a-43f7-b73b-353946e09a85 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:18.390346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-47a6347b-a92d-47da-a49a-9ff45111c559 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:18.390346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:18.390346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:18.390346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:18.390346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:18.390346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:18.390346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:18.390346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:18.390346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:18.390346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:18.390346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:18.390346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:18.390346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:18.390346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:18.390346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:18.390346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:18.391347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:18.391347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:18.391347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:18.391347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:18.391347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:18.391347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:18.391347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:18.391347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:18.391347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:18.391347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:18.391347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:18.391347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:18.391347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:18.391347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:18.391347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:18.391347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:18.391347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:18.392240 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:18.392240 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:18.392240 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:18.392240 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 221/443] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:18.392877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c3988-3880-488b-99aa-95fcdba8e526 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:18.392877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c3988-3880-488b-99aa-95fcdba8e526 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:18.393669 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c3988-3880-488b-99aa-95fcdba8e526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:18.393909 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 223/444] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:18.394964 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d22a4-131a-43f7-b73b-353946e09a85 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:18.415265 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d31c3988-3880-488b-99aa-95fcdba8e526 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:18.416449 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d31c3988-3880-488b-99aa-95fcdba8e526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:18.465099 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:18.481425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 222/445] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:52:18.482221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c3988-3880-488b-99aa-95fcdba8e526 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:18.496137 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d22a4-131a-43f7-b73b-353946e09a85 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:18.534611 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248a271-6f1a-4107-b8cd-9f83c45388e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:18.535280 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248a271-6f1a-4107-b8cd-9f83c45388e8 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:18.535552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248a271-6f1a-4107-b8cd-9f83c45388e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:18.535819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248a271-6f1a-4107-b8cd-9f83c45388e8 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:18.536543 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248a271-6f1a-4107-b8cd-9f83c45388e8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:18.536887 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248a271-6f1a-4107-b8cd-9f83c45388e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:18.538314 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248a271-6f1a-4107-b8cd-9f83c45388e8 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:18.561379 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 224/446] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Oct 20 05:52:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:18.561379 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248a271-6f1a-4107-b8cd-9f83c45388e8 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:18.587321 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c3988-3880-488b-99aa-95fcdba8e526 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:18.613205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:18.623006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf542181-062d-4f21-839e-681107e6c093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:18.623621 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf542181-062d-4f21-839e-681107e6c093 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:18.623899 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf542181-062d-4f21-839e-681107e6c093 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:18.624171 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf542181-062d-4f21-839e-681107e6c093 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:18.624857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf542181-062d-4f21-839e-681107e6c093 None admin] RBAC: Adding query filter params (name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:18.625162 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf542181-062d-4f21-839e-681107e6c093 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:18.626549 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf542181-062d-4f21-839e-681107e6c093 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:18.640022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 223/447] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Oct 20 05:52:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:18.640022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf542181-062d-4f21-839e-681107e6c093 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:18.670996 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248a271-6f1a-4107-b8cd-9f83c45388e8 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:18.687945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:18.710268 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b8a6f-b2a5-43c9-981c-37e603a8a4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:18.710975 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b8a6f-b2a5-43c9-981c-37e603a8a4ee None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:18.711249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b8a6f-b2a5-43c9-981c-37e603a8a4ee None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:18.711519 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b8a6f-b2a5-43c9-981c-37e603a8a4ee None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:18.712142 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b8a6f-b2a5-43c9-981c-37e603a8a4ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:18.712437 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b8a6f-b2a5-43c9-981c-37e603a8a4ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:18.713831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b8a6f-b2a5-43c9-981c-37e603a8a4ee None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:18.726201 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 225/448] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:18.727019 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b8a6f-b2a5-43c9-981c-37e603a8a4ee None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:18.741743 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf542181-062d-4f21-839e-681107e6c093 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:18.775494 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6933cb-0647-4ee7-9a2a-e167b59cb602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:18.776122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6933cb-0647-4ee7-9a2a-e167b59cb602 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:18.776396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6933cb-0647-4ee7-9a2a-e167b59cb602 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:18.776670 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6933cb-0647-4ee7-9a2a-e167b59cb602 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:18.784433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6933cb-0647-4ee7-9a2a-e167b59cb602 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:18.786069 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6933cb-0647-4ee7-9a2a-e167b59cb602 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:18.796826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-1e6933cb-0647-4ee7-9a2a-e167b59cb602 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 20 05:52:18.796826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:18.796826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:18.796826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:18.796826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:18.796826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:18.796826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:18.796826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:18.796826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:18.796826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:18.796826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:18.796826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:18.796826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:52:18.796826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:52:18.796826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:52:18.796826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:52:18.797877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:18.797877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:18.797877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:52:18.797877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:18.797877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:52:18.797877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:18.797877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:18.797877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:18.797877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:18.797877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:18.797877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:18.797877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:18.797877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:18.797877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:18.797877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:18.797877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:18.797877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:18.797877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:18.798963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:18.798963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:18.798963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:52:18.798963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:52:18.798963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:52:18.798963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:52:18.798963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:52:18.798963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:52:18.798963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 20 05:52:18.798963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:18.798963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 224/449] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:52:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:18.798963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6933cb-0647-4ee7-9a2a-e167b59cb602 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:18.837451 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b8a6f-b2a5-43c9-981c-37e603a8a4ee None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:18.847471 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:18.870729 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35923dea-2dfd-424c-bf87-a07462521986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:18.871338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35923dea-2dfd-424c-bf87-a07462521986 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:18.871621 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35923dea-2dfd-424c-bf87-a07462521986 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:18.871896 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35923dea-2dfd-424c-bf87-a07462521986 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:18.898698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6933cb-0647-4ee7-9a2a-e167b59cb602 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:18.907116 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:18.931466 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076cdac-9af3-453e-a281-2c2657a0f312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:18.932098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076cdac-9af3-453e-a281-2c2657a0f312 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:18.932370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076cdac-9af3-453e-a281-2c2657a0f312 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:18.932644 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076cdac-9af3-453e-a281-2c2657a0f312 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:18.938741 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35923dea-2dfd-424c-bf87-a07462521986 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:18.940726 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35923dea-2dfd-424c-bf87-a07462521986 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:18.943318 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076cdac-9af3-453e-a281-2c2657a0f312 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:18.944759 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076cdac-9af3-453e-a281-2c2657a0f312 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:18.958404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-3076cdac-9af3-453e-a281-2c2657a0f312 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:18.958404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:18.958404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:18.958404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:18.958404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:18.958404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:18.958404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:18.958404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:18.958404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:18.958404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:18.958404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:18.958404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:18.958404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:18.958404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:18.958404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:18.958404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:18.959438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:18.959438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:18.959438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:18.959438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:18.959438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:18.959438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:18.959438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:18.959438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:18.959438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:18.959438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:18.959438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:18.959438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:18.959438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:18.959438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:18.959438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:18.959438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:18.959438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:18.965393 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:18.965393 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:18.965393 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:18.965393 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 225/450] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:18.965393 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076cdac-9af3-453e-a281-2c2657a0f312 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:19.037223 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 226/451] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:52:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 05:52:19.038058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35923dea-2dfd-424c-bf87-a07462521986 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:19.048999 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:19.064364 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076cdac-9af3-453e-a281-2c2657a0f312 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:19.097245 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42042bc-f784-40d0-874f-ce1a0276bace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:19.097906 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42042bc-f784-40d0-874f-ce1a0276bace None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:19.098246 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42042bc-f784-40d0-874f-ce1a0276bace None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:19.098579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42042bc-f784-40d0-874f-ce1a0276bace None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:19.105020 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42042bc-f784-40d0-874f-ce1a0276bace None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:19.106447 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42042bc-f784-40d0-874f-ce1a0276bace None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:19.117576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-e42042bc-f784-40d0-874f-ce1a0276bace None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:19.117576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:19.117576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:19.117576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:19.117576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:19.117576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:19.117576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:19.117576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:19.117576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:19.117576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:19.117576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:19.117576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:19.117576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:52:19.117576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:52:19.117576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:52:19.118590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:52:19.118590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:19.118590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:19.118590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:19.118590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:19.118590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:19.118590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:19.118590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:19.118590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:19.118590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:19.118590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:19.118590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:19.118590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:19.118590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:19.118590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:19.118590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:19.118590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:19.119433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:52:19.119433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:52:19.119433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:52:19.119433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:52:19.119433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:52:19.119433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:52:19.119433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:19.119433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:19.119433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 226/452] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:19.119433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42042bc-f784-40d0-874f-ce1a0276bace None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:19.139002 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35923dea-2dfd-424c-bf87-a07462521986 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:19.140880 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:19.173338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582482de-2011-4d10-85a5-fe028c25e2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:19.173966 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582482de-2011-4d10-85a5-fe028c25e2de None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:19.174958 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582482de-2011-4d10-85a5-fe028c25e2de None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:19.175232 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582482de-2011-4d10-85a5-fe028c25e2de None admin] PATH_INFO: `/v3/roles/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:19.185063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582482de-2011-4d10-85a5-fe028c25e2de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:52:19.189860 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-582482de-2011-4d10-85a5-fe028c25e2de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:52:19.189860 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:19.189860 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:19.189860 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:19.189860 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:19.189860 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:19.189860 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:19.189860 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:19.189860 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:19.189860 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:19.189860 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:19.189860 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:19.189860 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:52:19.189860 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:52:19.189860 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:52:19.189860 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:52:19.190976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:52:19.190976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:52:19.190976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:52:19.190976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:52:19.190976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:19.190976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:19.190976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:19.190976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:19.190976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:19.190976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:19.190976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:19.190976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:19.190976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:19.190976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:19.190976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:19.190976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:19.191754 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:19.191754 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:19.191754 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:19.191754 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:19.191754 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:52:19.191754 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:52:19.191754 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:52:19.191754 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:52:19.191754 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:52:19.191754 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:52:19.191754 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:52:19.191754 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:19.191754 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 227/453] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Oct 20 05:52:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:19.192418 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582482de-2011-4d10-85a5-fe028c25e2de None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:19.219808 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42042bc-f784-40d0-874f-ce1a0276bace None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:19.253554 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175e849-8629-499b-995e-dff737309c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:19.254132 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175e849-8629-499b-995e-dff737309c0f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:19.254402 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175e849-8629-499b-995e-dff737309c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:19.254674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175e849-8629-499b-995e-dff737309c0f None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:19.255378 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175e849-8629-499b-995e-dff737309c0f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:19.255679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175e849-8629-499b-995e-dff737309c0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:19.257122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175e849-8629-499b-995e-dff737309c0f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:19.262740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:19.275409 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 227/454] 10.222.0.20 () {56 vars in 1137 bytes} [Thu Oct 20 05:52:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:19.276291 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175e849-8629-499b-995e-dff737309c0f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:19.295419 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582482de-2011-4d10-85a5-fe028c25e2de None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:19.328990 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6267fcc-618a-4cf3-a5d2-b8efbf24a206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:19.329597 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6267fcc-618a-4cf3-a5d2-b8efbf24a206 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:19.329875 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6267fcc-618a-4cf3-a5d2-b8efbf24a206 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:19.330140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6267fcc-618a-4cf3-a5d2-b8efbf24a206 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:19.331312 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6267fcc-618a-4cf3-a5d2-b8efbf24a206 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:19.331610 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6267fcc-618a-4cf3-a5d2-b8efbf24a206 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:19.333003 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6267fcc-618a-4cf3-a5d2-b8efbf24a206 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:19.344061 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 228/455] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:19.344874 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6267fcc-618a-4cf3-a5d2-b8efbf24a206 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:19.356144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:19.377012 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175e849-8629-499b-995e-dff737309c0f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:19.411434 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5657c-0c94-4055-997d-4171eb3a7856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:19.412064 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5657c-0c94-4055-997d-4171eb3a7856 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:19.412345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5657c-0c94-4055-997d-4171eb3a7856 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:19.412617 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5657c-0c94-4055-997d-4171eb3a7856 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:19.422400 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5657c-0c94-4055-997d-4171eb3a7856 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:19.423877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5657c-0c94-4055-997d-4171eb3a7856 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:19.438197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-d4d5657c-0c94-4055-997d-4171eb3a7856 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:19.438197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:19.438197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:19.438197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:19.438197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:19.438197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:19.438197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:19.438197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:19.438197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:19.438197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:19.438197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:19.438197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:19.438197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:19.438197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:19.438197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:19.438197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:19.439244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:19.439244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:19.439244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:19.439244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:19.439244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:19.439244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:19.439244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:19.439244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:19.439244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:19.439244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:19.439244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:19.439244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:19.439244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:19.439244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:19.439244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:19.439244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:19.439244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:19.440269 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:19.440269 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:19.440269 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:19.443829 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 228/456] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:19.444658 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5657c-0c94-4055-997d-4171eb3a7856 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:19.446733 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6267fcc-618a-4cf3-a5d2-b8efbf24a206 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:19.477539 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:19.487878 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecaddf5-bca1-4933-95fb-3c76ace53185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:19.488490 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecaddf5-bca1-4933-95fb-3c76ace53185 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:19.488785 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecaddf5-bca1-4933-95fb-3c76ace53185 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:19.489060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecaddf5-bca1-4933-95fb-3c76ace53185 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:19.489746 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecaddf5-bca1-4933-95fb-3c76ace53185 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:19.490058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecaddf5-bca1-4933-95fb-3c76ace53185 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:19.491484 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecaddf5-bca1-4933-95fb-3c76ace53185 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:19.504669 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 229/457] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Oct 20 05:52:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:19.505570 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecaddf5-bca1-4933-95fb-3c76ace53185 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:19.543458 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5657c-0c94-4055-997d-4171eb3a7856 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:19.570512 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:19.576640 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974333f-0017-4ad2-8c27-d7456cca501e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:19.577271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974333f-0017-4ad2-8c27-d7456cca501e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:19.577547 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974333f-0017-4ad2-8c27-d7456cca501e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:19.577821 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974333f-0017-4ad2-8c27-d7456cca501e None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:19.578473 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974333f-0017-4ad2-8c27-d7456cca501e None admin] RBAC: Adding query filter params (name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:19.578777 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974333f-0017-4ad2-8c27-d7456cca501e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:19.580150 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974333f-0017-4ad2-8c27-d7456cca501e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:19.588637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 229/458] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Oct 20 05:52:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:19.589532 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974333f-0017-4ad2-8c27-d7456cca501e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:19.608113 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecaddf5-bca1-4933-95fb-3c76ace53185 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:19.641763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdc895-2b40-4eb0-82a8-894113e59a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:19.642366 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdc895-2b40-4eb0-82a8-894113e59a77 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:19.642638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdc895-2b40-4eb0-82a8-894113e59a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:19.642904 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdc895-2b40-4eb0-82a8-894113e59a77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:19.687156 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974333f-0017-4ad2-8c27-d7456cca501e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:19.699520 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:19.710611 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cdc895-2b40-4eb0-82a8-894113e59a77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:19.712068 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cdc895-2b40-4eb0-82a8-894113e59a77 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:19.720295 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09cc84-a325-4e20-8768-28b1ea35932e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:19.720950 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09cc84-a325-4e20-8768-28b1ea35932e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:19.721745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09cc84-a325-4e20-8768-28b1ea35932e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:19.722014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09cc84-a325-4e20-8768-28b1ea35932e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:19.729247 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d09cc84-a325-4e20-8768-28b1ea35932e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:19.730748 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d09cc84-a325-4e20-8768-28b1ea35932e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:19.742170 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-0d09cc84-a325-4e20-8768-28b1ea35932e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 20 05:52:19.742170 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:19.742170 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:19.742170 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:19.742170 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:19.742170 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:19.742170 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:19.742170 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:19.742170 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:19.742170 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:19.742170 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:19.742170 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:19.742170 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:52:19.742170 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:52:19.742170 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:52:19.742170 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:52:19.743310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:19.743310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:19.743310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:52:19.743310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:19.743310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:52:19.743310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:19.743310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:19.743310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:19.743310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:19.743310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:19.743310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:19.743310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:19.743310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:19.743310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:19.743310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:19.743310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:19.743310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:19.743310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:19.744446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:19.744446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:19.744446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:52:19.744446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:52:19.744446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:52:19.744446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:52:19.744446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:52:19.744446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:52:19.744446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 20 05:52:19.744446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:19.744446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 230/459] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:52:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:19.744446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09cc84-a325-4e20-8768-28b1ea35932e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:19.795000 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:19.814734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 230/460] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:52:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 05:52:19.816075 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdc895-2b40-4eb0-82a8-894113e59a77 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:19.842290 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09cc84-a325-4e20-8768-28b1ea35932e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:19.875634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390d962-bda1-4d91-9bc2-120599484b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:19.876163 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390d962-bda1-4d91-9bc2-120599484b97 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:19.876444 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390d962-bda1-4d91-9bc2-120599484b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:19.876744 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390d962-bda1-4d91-9bc2-120599484b97 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:19.877379 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390d962-bda1-4d91-9bc2-120599484b97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:19.878168 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390d962-bda1-4d91-9bc2-120599484b97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:19.879539 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390d962-bda1-4d91-9bc2-120599484b97 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:19.891084 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 231/461] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:19.891858 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390d962-bda1-4d91-9bc2-120599484b97 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:19.905842 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:19.917148 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdc895-2b40-4eb0-82a8-894113e59a77 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:19.951784 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132ba82-6216-4726-931d-03785cc9dc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:19.952442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132ba82-6216-4726-931d-03785cc9dc97 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:19.952758 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132ba82-6216-4726-931d-03785cc9dc97 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:19.953033 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132ba82-6216-4726-931d-03785cc9dc97 None admin] PATH_INFO: `/v3/projects/0ee65b9a8229495bbe97318f060d6342/users/eeeab506ab5c4d6386fc899086f0abf5/roles/afd859d2deed4d51aa3f6326920441c8` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:19.968183 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132ba82-6216-4726-931d-03785cc9dc97 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ee65b9a8229495bbe97318f060d6342, user_id=eeeab506ab5c4d6386fc899086f0abf5, role_id=afd859d2deed4d51aa3f6326920441c8)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:19.969651 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132ba82-6216-4726-931d-03785cc9dc97 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:19.982216 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 231/462] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:52:19 2022] PUT /identity/v3/projects/0ee65b9a8229495bbe97318f060d6342/users/eeeab506ab5c4d6386fc899086f0abf5/roles/afd859d2deed4d51aa3f6326920441c8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:52:19.983184 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132ba82-6216-4726-931d-03785cc9dc97 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:20.016514 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390d962-bda1-4d91-9bc2-120599484b97 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:20.033147 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:20.064290 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcdb4c-5a36-41fd-8d38-334a99513d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:20.066405 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcdb4c-5a36-41fd-8d38-334a99513d28 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:20.066786 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcdb4c-5a36-41fd-8d38-334a99513d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:20.067155 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcdb4c-5a36-41fd-8d38-334a99513d28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:20.081475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebcdb4c-5a36-41fd-8d38-334a99513d28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:20.083006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebcdb4c-5a36-41fd-8d38-334a99513d28 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:20.093605 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132ba82-6216-4726-931d-03785cc9dc97 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:20.101052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-aebcdb4c-5a36-41fd-8d38-334a99513d28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:20.101052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:20.101052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:20.101052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:20.101052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:20.101052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:20.101052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:20.101052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:20.101052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:20.101052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:20.101052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:20.101052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:20.101052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:20.101052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:20.101052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:20.101052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:20.102028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:20.102028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:20.102028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:20.102028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:20.102028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:20.102028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:20.102028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:20.102028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:20.102028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:20.102028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:20.102028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:20.102028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:20.102028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:20.102028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:20.102028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:20.102028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:20.102028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:20.102866 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:20.102866 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:20.102866 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:20.102866 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 232/463] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:20.103094 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcdb4c-5a36-41fd-8d38-334a99513d28 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:20.137465 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0c2c5-c9eb-4c32-9d8d-a93fc88b6710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:20.138088 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0c2c5-c9eb-4c32-9d8d-a93fc88b6710 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:20.138366 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0c2c5-c9eb-4c32-9d8d-a93fc88b6710 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:20.138637 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0c2c5-c9eb-4c32-9d8d-a93fc88b6710 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:20.139370 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc0c2c5-c9eb-4c32-9d8d-a93fc88b6710 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:20.139933 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc0c2c5-c9eb-4c32-9d8d-a93fc88b6710 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:20.141441 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc0c2c5-c9eb-4c32-9d8d-a93fc88b6710 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:20.152711 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:20.166213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 232/464] 10.222.0.20 () {56 vars in 1137 bytes} [Thu Oct 20 05:52:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:20.167049 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0c2c5-c9eb-4c32-9d8d-a93fc88b6710 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:20.220549 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcdb4c-5a36-41fd-8d38-334a99513d28 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:20.256119 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5063d9-b91f-41fe-bf0e-cb1bd87f0925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:20.256881 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5063d9-b91f-41fe-bf0e-cb1bd87f0925 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:20.257156 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5063d9-b91f-41fe-bf0e-cb1bd87f0925 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:20.257436 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5063d9-b91f-41fe-bf0e-cb1bd87f0925 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:20.269262 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5063d9-b91f-41fe-bf0e-cb1bd87f0925 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:20.270714 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5063d9-b91f-41fe-bf0e-cb1bd87f0925 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:20.275576 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0c2c5-c9eb-4c32-9d8d-a93fc88b6710 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:20.287465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-cd5063d9-b91f-41fe-bf0e-cb1bd87f0925 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:20.287465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:20.287465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:20.287465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:20.287465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:20.287465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:20.287465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:20.287465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:20.287465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:20.287465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:20.287465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:20.287465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:20.287465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:20.287465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:20.287465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:20.287465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:20.288537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:20.288537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:20.288537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:20.288537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:20.288537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:20.288537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:20.288537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:20.288537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:20.288537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:20.288537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:20.288537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:20.288537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:20.288537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:20.288537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:20.288537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:20.288537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:20.288537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:20.289613 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:20.289613 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:20.289613 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:20.291305 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 233/465] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:20.292090 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5063d9-b91f-41fe-bf0e-cb1bd87f0925 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:20.311225 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:20.320500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1197e-9672-4f8a-b068-d54c9d007672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:20.321224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1197e-9672-4f8a-b068-d54c9d007672 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:20.321506 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1197e-9672-4f8a-b068-d54c9d007672 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:20.321778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1197e-9672-4f8a-b068-d54c9d007672 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:20.328627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1197e-9672-4f8a-b068-d54c9d007672 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:20.330115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1197e-9672-4f8a-b068-d54c9d007672 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:20.341799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-61c1197e-9672-4f8a-b068-d54c9d007672 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:20.341799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:20.341799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:20.341799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:20.341799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:20.341799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:20.341799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:20.341799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:20.341799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:20.341799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:20.341799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:20.341799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:20.341799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:52:20.341799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:52:20.341799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:52:20.342670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:52:20.342670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:20.342670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:20.342670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:20.342670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:20.342670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:20.342670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:20.342670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:20.342670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:20.342670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:20.342670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:20.342670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:20.342670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:20.342670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:20.342670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:20.342670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:20.342670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:20.343502 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:52:20.343502 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:52:20.343502 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:52:20.343502 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:52:20.343502 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:52:20.343502 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:52:20.343502 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:20.343502 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:20.343502 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 233/466] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:20.344950 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1197e-9672-4f8a-b068-d54c9d007672 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:20.403925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5063d9-b91f-41fe-bf0e-cb1bd87f0925 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:20.409666 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:20.439506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e633a-1cc4-4e31-a400-5dff86907ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:20.440718 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e633a-1cc4-4e31-a400-5dff86907ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:20.440997 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e633a-1cc4-4e31-a400-5dff86907ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:20.441264 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e633a-1cc4-4e31-a400-5dff86907ef9 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:20.441892 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e633a-1cc4-4e31-a400-5dff86907ef9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:20.442194 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e633a-1cc4-4e31-a400-5dff86907ef9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:20.443578 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e633a-1cc4-4e31-a400-5dff86907ef9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:20.457472 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1197e-9672-4f8a-b068-d54c9d007672 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:20.460333 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 234/467] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:20.461361 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e633a-1cc4-4e31-a400-5dff86907ef9 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:20.496500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcffb95-4d89-4245-81da-0fa61c849fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:20.497245 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcffb95-4d89-4245-81da-0fa61c849fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:20.497527 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcffb95-4d89-4245-81da-0fa61c849fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:20.497809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcffb95-4d89-4245-81da-0fa61c849fd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:20.542921 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:20.569125 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e633a-1cc4-4e31-a400-5dff86907ef9 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:20.571938 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcffb95-4d89-4245-81da-0fa61c849fd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:20.573499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcffb95-4d89-4245-81da-0fa61c849fd5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:20.607982 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40665b-37cf-4837-bdf2-5c7f5e3d3eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:20.608613 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40665b-37cf-4837-bdf2-5c7f5e3d3eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:20.608969 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40665b-37cf-4837-bdf2-5c7f5e3d3eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:20.609241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40665b-37cf-4837-bdf2-5c7f5e3d3eb0 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:20.609883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40665b-37cf-4837-bdf2-5c7f5e3d3eb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:20.610189 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40665b-37cf-4837-bdf2-5c7f5e3d3eb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:20.611724 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40665b-37cf-4837-bdf2-5c7f5e3d3eb0 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:20.624441 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 235/468] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:20.625399 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40665b-37cf-4837-bdf2-5c7f5e3d3eb0 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:20.642912 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:20.694439 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 234/469] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:52:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 05:52:20.694439 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcffb95-4d89-4245-81da-0fa61c849fd5 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:20.735670 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40665b-37cf-4837-bdf2-5c7f5e3d3eb0 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:20.770998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6f4d3-0f0f-47e5-af8f-669b7ad68644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:20.771608 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6f4d3-0f0f-47e5-af8f-669b7ad68644 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:20.771878 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6f4d3-0f0f-47e5-af8f-669b7ad68644 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:20.772145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6f4d3-0f0f-47e5-af8f-669b7ad68644 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:20.772848 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6f4d3-0f0f-47e5-af8f-669b7ad68644 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:20.773148 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6f4d3-0f0f-47e5-af8f-669b7ad68644 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:20.774569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6f4d3-0f0f-47e5-af8f-669b7ad68644 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:20.777050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:20.792519 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 236/470] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Oct 20 05:52:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:20.792519 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6f4d3-0f0f-47e5-af8f-669b7ad68644 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:20.810253 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcffb95-4d89-4245-81da-0fa61c849fd5 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:20.845382 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e9b05-c969-40d8-83b7-35641fce2008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:20.846019 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e9b05-c969-40d8-83b7-35641fce2008 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:20.846849 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e9b05-c969-40d8-83b7-35641fce2008 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:20.847127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e9b05-c969-40d8-83b7-35641fce2008 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:20.855902 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e9b05-c969-40d8-83b7-35641fce2008 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:20.857442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e9b05-c969-40d8-83b7-35641fce2008 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:20.870734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-645e9b05-c969-40d8-83b7-35641fce2008 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 20 05:52:20.870734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:20.870734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:20.870734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:20.870734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:20.870734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:20.870734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:20.870734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:20.870734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:20.870734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:20.870734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:20.870734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:20.870734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:52:20.870734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:52:20.870734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:52:20.870734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:52:20.877000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:20.877000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:20.877000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:52:20.877000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:20.877000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:52:20.877000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:20.877000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:20.877000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:20.877000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:20.877000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:20.877000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:20.877000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:20.877000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:20.877000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:20.877000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:20.877000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:20.877000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:20.877000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:20.878197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:20.878197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:20.878197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:52:20.878197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:52:20.878197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:52:20.878197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:52:20.878197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:52:20.878197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:52:20.878197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 20 05:52:20.878197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:20.878197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 235/471] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Oct 20 05:52:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:20.878197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e9b05-c969-40d8-83b7-35641fce2008 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:20.896067 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:20.905058 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6f4d3-0f0f-47e5-af8f-669b7ad68644 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:20.944885 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc120bf8-9bbf-49f4-8638-b7d9e88a0410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:20.945439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc120bf8-9bbf-49f4-8638-b7d9e88a0410 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:20.945711 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc120bf8-9bbf-49f4-8638-b7d9e88a0410 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:20.945978 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc120bf8-9bbf-49f4-8638-b7d9e88a0410 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:20.952589 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc120bf8-9bbf-49f4-8638-b7d9e88a0410 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:20.954059 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc120bf8-9bbf-49f4-8638-b7d9e88a0410 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:20.965634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-cc120bf8-9bbf-49f4-8638-b7d9e88a0410 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:20.965634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:20.965634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:20.965634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:20.965634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:20.965634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:20.965634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:20.965634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:20.965634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:20.965634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:20.965634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:20.965634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:20.965634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:52:20.965634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:52:20.965634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:52:20.966483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:52:20.966483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:20.966483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:20.966483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:20.966483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:20.966483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:20.966483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:20.966483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:20.966483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:20.966483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:20.966483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:20.966483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:20.966483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:20.966483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:20.966483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:20.966483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:20.966483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:20.967945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:52:20.967945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:52:20.967945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:52:20.967945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:52:20.967945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:52:20.967945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:52:20.967945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:20.967945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:20.967945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 237/472] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:20.967945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc120bf8-9bbf-49f4-8638-b7d9e88a0410 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:20.987949 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e9b05-c969-40d8-83b7-35641fce2008 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:21.023176 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b032bb0-653c-466f-96eb-f8756ed38275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:21.023815 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b032bb0-653c-466f-96eb-f8756ed38275 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:21.024096 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b032bb0-653c-466f-96eb-f8756ed38275 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:21.024366 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b032bb0-653c-466f-96eb-f8756ed38275 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:21.028593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:21.037598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b032bb0-653c-466f-96eb-f8756ed38275 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:21.039056 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b032bb0-653c-466f-96eb-f8756ed38275 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:21.053859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-7b032bb0-653c-466f-96eb-f8756ed38275 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:21.053859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:21.053859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:21.053859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:21.053859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:21.053859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:21.053859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:21.053859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:21.053859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:21.053859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:21.053859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:21.053859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:21.053859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:21.053859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:21.053859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:21.053859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:21.054724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:21.054724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:21.054724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:21.054724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:21.054724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:21.054724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:21.054724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:21.054724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:21.054724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:21.054724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:21.054724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:21.054724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:21.054724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:21.054724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:21.054724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:21.054724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:21.054724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:21.055579 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:21.055579 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:21.055579 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:21.055579 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 236/473] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:21.055799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b032bb0-653c-466f-96eb-f8756ed38275 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:21.081344 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc120bf8-9bbf-49f4-8638-b7d9e88a0410 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:21.118217 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4fa68-c035-4961-bc20-6127ce93deb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:21.119119 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4fa68-c035-4961-bc20-6127ce93deb9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:21.119407 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4fa68-c035-4961-bc20-6127ce93deb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:21.119683 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4fa68-c035-4961-bc20-6127ce93deb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:21.120576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4fa68-c035-4961-bc20-6127ce93deb9 None admin] RBAC: Adding query filter params (role.id=afd859d2deed4d51aa3f6326920441c8, scope.project.id=0ee65b9a8229495bbe97318f060d6342, user.id=ae2c1c886cd94d9f90082f66e7e8c9f4) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:21.121223 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4fa68-c035-4961-bc20-6127ce93deb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:21.122960 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4fa68-c035-4961-bc20-6127ce93deb9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:21.129019 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:21.136901 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 238/474] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:52:20 2022] GET /identity/v3/role_assignments?user.id=ae2c1c886cd94d9f90082f66e7e8c9f4&scope.project.id=0ee65b9a8229495bbe97318f060d6342&role.id=afd859d2deed4d51aa3f6326920441c8 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:21.136901 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4fa68-c035-4961-bc20-6127ce93deb9 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:21.170143 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b032bb0-653c-466f-96eb-f8756ed38275 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:21.210737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2e1f5-856a-441d-be79-5d4989ab463b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:21.211273 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2e1f5-856a-441d-be79-5d4989ab463b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:21.211590 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2e1f5-856a-441d-be79-5d4989ab463b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:21.211938 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2e1f5-856a-441d-be79-5d4989ab463b None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:21.212869 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe2e1f5-856a-441d-be79-5d4989ab463b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:21.213173 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe2e1f5-856a-441d-be79-5d4989ab463b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:21.214642 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe2e1f5-856a-441d-be79-5d4989ab463b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:21.235321 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 237/475] 10.222.0.20 () {56 vars in 1139 bytes} [Thu Oct 20 05:52:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:21.236191 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2e1f5-856a-441d-be79-5d4989ab463b None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:21.276977 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4fa68-c035-4961-bc20-6127ce93deb9 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:21.305138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:21.326401 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe0e9b-60bc-4968-8345-edbf67b5eb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:21.327001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe0e9b-60bc-4968-8345-edbf67b5eb95 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:21.327288 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe0e9b-60bc-4968-8345-edbf67b5eb95 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:21.327557 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe0e9b-60bc-4968-8345-edbf67b5eb95 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:21.328529 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebe0e9b-60bc-4968-8345-edbf67b5eb95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:21.328733 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebe0e9b-60bc-4968-8345-edbf67b5eb95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:21.330121 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebe0e9b-60bc-4968-8345-edbf67b5eb95 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:21.349001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 239/476] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Oct 20 05:52:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:21.349290 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe0e9b-60bc-4968-8345-edbf67b5eb95 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:21.351516 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2e1f5-856a-441d-be79-5d4989ab463b None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:21.382802 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:21.395750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafb904-50b9-4052-814a-977b75322c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:21.396369 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafb904-50b9-4052-814a-977b75322c43 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:21.396643 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafb904-50b9-4052-814a-977b75322c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:21.396973 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafb904-50b9-4052-814a-977b75322c43 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:21.398074 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecafb904-50b9-4052-814a-977b75322c43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:21.398376 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecafb904-50b9-4052-814a-977b75322c43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:21.400228 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecafb904-50b9-4052-814a-977b75322c43 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:21.417107 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 238/477] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:21.418010 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafb904-50b9-4052-814a-977b75322c43 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:21.464443 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe0e9b-60bc-4968-8345-edbf67b5eb95 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:21.501906 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f165a-bc42-4137-a848-49f6e094135f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:21.503623 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f165a-bc42-4137-a848-49f6e094135f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:21.504142 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f165a-bc42-4137-a848-49f6e094135f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:21.504729 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f165a-bc42-4137-a848-49f6e094135f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:21.525011 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafb904-50b9-4052-814a-977b75322c43 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:21.560397 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b96d82d-8a33-4edb-a6ff-3f3a6c93f52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:21.561423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96d82d-8a33-4edb-a6ff-3f3a6c93f52d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:21.561629 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96d82d-8a33-4edb-a6ff-3f3a6c93f52d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:21.561891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96d82d-8a33-4edb-a6ff-3f3a6c93f52d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:21.562299 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:21.562622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b96d82d-8a33-4edb-a6ff-3f3a6c93f52d None admin] RBAC: Adding query filter params (role.id=afd859d2deed4d51aa3f6326920441c8, scope.project.id=0ee65b9a8229495bbe97318f060d6342, user.id=cfb00a74a99e47c1ac938f07d26a4bc8) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:21.563217 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b96d82d-8a33-4edb-a6ff-3f3a6c93f52d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:21.564507 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b96d82d-8a33-4edb-a6ff-3f3a6c93f52d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:21.576735 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 239/478] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:52:21 2022] GET /identity/v3/role_assignments?user.id=cfb00a74a99e47c1ac938f07d26a4bc8&scope.project.id=0ee65b9a8229495bbe97318f060d6342&role.id=afd859d2deed4d51aa3f6326920441c8 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:21.576735 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b96d82d-8a33-4edb-a6ff-3f3a6c93f52d None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:21.601434 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7f165a-bc42-4137-a848-49f6e094135f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:21.612768 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7f165a-bc42-4137-a848-49f6e094135f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:21.635711 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:21.689615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b96d82d-8a33-4edb-a6ff-3f3a6c93f52d None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:21.731482 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac422fb1-c6a5-4927-ba9e-1a851f979fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:21.732046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac422fb1-c6a5-4927-ba9e-1a851f979fdb None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:21.732318 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac422fb1-c6a5-4927-ba9e-1a851f979fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:21.732584 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac422fb1-c6a5-4927-ba9e-1a851f979fdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:21.748908 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac422fb1-c6a5-4927-ba9e-1a851f979fdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:21.750380 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac422fb1-c6a5-4927-ba9e-1a851f979fdb None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:21.773700 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-ac422fb1-c6a5-4927-ba9e-1a851f979fdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:21.773700 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:21.773700 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:21.773700 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:21.773700 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:21.773700 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:21.773700 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:21.773700 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:21.773700 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:21.773700 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:21.773700 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:21.773700 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:21.773700 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:52:21.773700 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:52:21.773700 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:52:21.774630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:52:21.774630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:21.774630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:21.774630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:21.774630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:21.774630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:21.774630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:21.774630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:21.774630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:21.774630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:21.774630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:21.774630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:21.774630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:21.774630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:21.774630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:21.774630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:21.774630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:21.775614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:52:21.775614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:52:21.775614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:52:21.775614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:52:21.775614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:52:21.775614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:52:21.775614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:21.775614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:21.785453 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 240/479] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:21.788087 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac422fb1-c6a5-4927-ba9e-1a851f979fdb None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:21.820254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 240/480] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:52:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 05:52:21.826157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f165a-bc42-4137-a848-49f6e094135f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:21.919296 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac422fb1-c6a5-4927-ba9e-1a851f979fdb None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:21.932917 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:21.935576 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:21.958101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f165a-bc42-4137-a848-49f6e094135f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:21.959925 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd930179-81eb-4d73-a2c6-34443fb0c8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:21.960737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd930179-81eb-4d73-a2c6-34443fb0c8f2 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:21.961006 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd930179-81eb-4d73-a2c6-34443fb0c8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:21.961277 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd930179-81eb-4d73-a2c6-34443fb0c8f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:21.962795 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd930179-81eb-4d73-a2c6-34443fb0c8f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:21.963103 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd930179-81eb-4d73-a2c6-34443fb0c8f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:21.964530 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd930179-81eb-4d73-a2c6-34443fb0c8f2 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:21.979281 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 241/481] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Oct 20 05:52:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:21.985329 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd930179-81eb-4d73-a2c6-34443fb0c8f2 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:22.001361 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c617146-7e5d-40ad-b844-75b9951055c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:22.003038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c617146-7e5d-40ad-b844-75b9951055c7 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:22.003589 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c617146-7e5d-40ad-b844-75b9951055c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:22.004058 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c617146-7e5d-40ad-b844-75b9951055c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:22.016081 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c617146-7e5d-40ad-b844-75b9951055c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:22.017944 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c617146-7e5d-40ad-b844-75b9951055c7 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:22.035373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-2c617146-7e5d-40ad-b844-75b9951055c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:22.035373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:22.035373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:22.035373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:22.035373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:22.035373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:22.035373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:22.035373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:22.035373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:22.035373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:22.035373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:22.035373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:22.035373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:22.035373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:22.035373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:22.035373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:22.036313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:22.036313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:22.036313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:22.036313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:22.036313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:22.036313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:22.036313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:22.036313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:22.036313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:22.036313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:22.036313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:22.036313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:22.036313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:22.036313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:22.036313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:22.036313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:22.036313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:22.037278 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:22.037278 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:22.037278 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:22.039470 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 241/482] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:22.045329 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c617146-7e5d-40ad-b844-75b9951055c7 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:22.094871 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd930179-81eb-4d73-a2c6-34443fb0c8f2 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:22.131890 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75150ffa-30b8-4224-9ea2-25f706ea0afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:22.132501 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75150ffa-30b8-4224-9ea2-25f706ea0afa None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:22.132806 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75150ffa-30b8-4224-9ea2-25f706ea0afa None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:22.133072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75150ffa-30b8-4224-9ea2-25f706ea0afa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:22.133792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75150ffa-30b8-4224-9ea2-25f706ea0afa None admin] RBAC: Adding query filter params (role.id=afd859d2deed4d51aa3f6326920441c8, scope.project.id=0ee65b9a8229495bbe97318f060d6342, user.id=3dcfe5244bd04c00b6dde1d6f795d4ec) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:22.134095 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75150ffa-30b8-4224-9ea2-25f706ea0afa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:22.135779 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75150ffa-30b8-4224-9ea2-25f706ea0afa None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:22.142420 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 242/483] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:52:21 2022] GET /identity/v3/role_assignments?user.id=3dcfe5244bd04c00b6dde1d6f795d4ec&scope.project.id=0ee65b9a8229495bbe97318f060d6342&role.id=afd859d2deed4d51aa3f6326920441c8 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:22.168828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c617146-7e5d-40ad-b844-75b9951055c7 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:22.183483 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a392e-2b0b-4cf4-a47f-33d5608f2b47 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:22.184001 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a392e-2b0b-4cf4-a47f-33d5608f2b47 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:22.184001 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a392e-2b0b-4cf4-a47f-33d5608f2b47 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:22.185361 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 243/484] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:52:22.191779 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbabada-21c4-4ce2-9cee-287c4922f31d None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:22.192099 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbabada-21c4-4ce2-9cee-287c4922f31d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:22.192319 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbabada-21c4-4ce2-9cee-287c4922f31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:22.214449 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-6dbabada-21c4-4ce2-9cee-287c4922f31d None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:22.216057 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6dbabada-21c4-4ce2-9cee-287c4922f31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:22.216588 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:22.232904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21dc2e3-52fe-4104-aa2a-27c662069a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:22.233901 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dc2e3-52fe-4104-aa2a-27c662069a82 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:22.234403 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dc2e3-52fe-4104-aa2a-27c662069a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:22.234874 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dc2e3-52fe-4104-aa2a-27c662069a82 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:22.235810 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21dc2e3-52fe-4104-aa2a-27c662069a82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:22.236666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21dc2e3-52fe-4104-aa2a-27c662069a82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:22.239827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21dc2e3-52fe-4104-aa2a-27c662069a82 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:22.265357 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 242/485] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:22.272726 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21dc2e3-52fe-4104-aa2a-27c662069a82 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:22.288199 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:22.374630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 244/486] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 05:52:22.423381 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21dc2e3-52fe-4104-aa2a-27c662069a82 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:22.459818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34961d-6386-446a-819a-060b3490c694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:22.460459 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34961d-6386-446a-819a-060b3490c694 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:22.460761 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34961d-6386-446a-819a-060b3490c694 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:22.461045 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34961d-6386-446a-819a-060b3490c694 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:22.468133 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc34961d-6386-446a-819a-060b3490c694 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:22.469607 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc34961d-6386-446a-819a-060b3490c694 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:22.481992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-bc34961d-6386-446a-819a-060b3490c694 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:22.481992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:22.481992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:22.481992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:22.481992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:22.481992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:22.481992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:22.481992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:22.481992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:22.481992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:22.481992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:22.481992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:22.481992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:52:22.481992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:52:22.481992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:52:22.482861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:52:22.482861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:22.482861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:22.482861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:22.482861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:22.482861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:22.482861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:22.482861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:22.482861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:22.482861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:22.482861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:22.482861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:22.482861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:22.482861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:22.482861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:22.482861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:22.482861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:22.484295 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:52:22.484295 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:52:22.484295 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:52:22.484295 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:52:22.484295 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:52:22.484295 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:52:22.484295 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:22.484295 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:22.484295 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 243/487] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:52:22.489532 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbabada-21c4-4ce2-9cee-287c4922f31d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:22.594689 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbabada-21c4-4ce2-9cee-287c4922f31d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:22.632467 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396db00e-f35c-4679-ad61-771229c075c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:22.633114 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396db00e-f35c-4679-ad61-771229c075c4 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:22.633397 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396db00e-f35c-4679-ad61-771229c075c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:22.633671 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396db00e-f35c-4679-ad61-771229c075c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:22.634360 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396db00e-f35c-4679-ad61-771229c075c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:22.634670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396db00e-f35c-4679-ad61-771229c075c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:22.636371 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396db00e-f35c-4679-ad61-771229c075c4 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:22.648900 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 245/488] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Oct 20 05:52:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:22.654665 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34961d-6386-446a-819a-060b3490c694 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:22.676260 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396db00e-f35c-4679-ad61-771229c075c4 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:22.724125 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:22.724857 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:22.802641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34961d-6386-446a-819a-060b3490c694 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:22.837231 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396db00e-f35c-4679-ad61-771229c075c4 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:22.864277 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e7b6e-4305-4919-8ba4-def862fdd4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:22.864905 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e7b6e-4305-4919-8ba4-def862fdd4fe None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:22.865179 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e7b6e-4305-4919-8ba4-def862fdd4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:22.865444 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e7b6e-4305-4919-8ba4-def862fdd4fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:22.866161 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e7b6e-4305-4919-8ba4-def862fdd4fe None admin] RBAC: Adding query filter params (role.id=afd859d2deed4d51aa3f6326920441c8, scope.project.id=0ee65b9a8229495bbe97318f060d6342, user.id=9072ce66e7174837aeb8c7898026fcce) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:22.866461 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e7b6e-4305-4919-8ba4-def862fdd4fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:22.867879 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e7b6e-4305-4919-8ba4-def862fdd4fe None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:22.883908 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 244/489] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:52:22 2022] GET /identity/v3/role_assignments?user.id=9072ce66e7174837aeb8c7898026fcce&scope.project.id=0ee65b9a8229495bbe97318f060d6342&role.id=afd859d2deed4d51aa3f6326920441c8 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:22.901186 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2aacf7-b21b-452c-a041-63d999e71acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:22.901857 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2aacf7-b21b-452c-a041-63d999e71acf None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:22.902132 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2aacf7-b21b-452c-a041-63d999e71acf None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:22.902401 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2aacf7-b21b-452c-a041-63d999e71acf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:22.921755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2aacf7-b21b-452c-a041-63d999e71acf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:52:22.927792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-2c2aacf7-b21b-452c-a041-63d999e71acf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:52:22.927792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:22.927792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:22.927792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:22.927792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:22.927792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:22.927792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:22.927792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:22.927792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:22.927792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:22.927792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:22.927792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:22.927792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:52:22.927792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:52:22.927792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:52:22.927792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:52:22.928786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:52:22.928786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:52:22.928786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:52:22.928786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:52:22.928786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:22.928786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:22.928786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:22.928786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:22.928786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:22.928786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:22.928786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:22.928786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:22.928786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:22.928786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:22.928786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:22.928786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:22.930427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:22.930427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:22.930427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:22.930427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:22.930427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:52:22.930427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:52:22.930427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:52:22.930427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:52:22.930427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:52:22.930427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:52:22.930427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:52:22.930427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:22.930427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 246/490] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Oct 20 05:52:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:22.935301 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e7b6e-4305-4919-8ba4-def862fdd4fe None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:23.084779 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e7b6e-4305-4919-8ba4-def862fdd4fe None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:23.101645 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:23.122620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230bb4c-76f8-4282-bada-b91fa2371f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:23.123236 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230bb4c-76f8-4282-bada-b91fa2371f12 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:23.123512 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230bb4c-76f8-4282-bada-b91fa2371f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:23.123782 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230bb4c-76f8-4282-bada-b91fa2371f12 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:23.124442 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8230bb4c-76f8-4282-bada-b91fa2371f12 None admin] RBAC: Adding query filter params (name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:23.124798 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8230bb4c-76f8-4282-bada-b91fa2371f12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:23.126161 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8230bb4c-76f8-4282-bada-b91fa2371f12 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:23.135728 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 245/491] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Oct 20 05:52:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:23.141008 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2aacf7-b21b-452c-a041-63d999e71acf None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:23.236344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:23.252806 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2aacf7-b21b-452c-a041-63d999e71acf None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:23.301904 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd90832-97c0-49bc-9198-3ed3e7932a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:23.301904 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd90832-97c0-49bc-9198-3ed3e7932a00 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:23.301904 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd90832-97c0-49bc-9198-3ed3e7932a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:23.301904 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd90832-97c0-49bc-9198-3ed3e7932a00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:23.320564 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd90832-97c0-49bc-9198-3ed3e7932a00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:23.322177 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd90832-97c0-49bc-9198-3ed3e7932a00 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:23.345159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-1dd90832-97c0-49bc-9198-3ed3e7932a00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:23.345159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:23.345159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:23.345159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:23.345159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:23.345159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:23.345159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:23.345159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:23.345159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:23.345159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:23.345159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:23.345159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:23.345159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:23.345159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:23.345159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:23.345159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:23.346059 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:23.346059 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:23.346059 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:23.346059 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:23.346059 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:23.346059 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:23.346059 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:23.346059 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:23.346059 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:23.346059 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:23.346059 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:23.346059 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:23.346059 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:23.346059 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:23.346059 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:23.346059 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:23.346059 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:23.348883 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:23.348883 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:23.348883 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:23.348883 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 247/492] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:52:23.352282 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230bb4c-76f8-4282-bada-b91fa2371f12 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:23.484350 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230bb4c-76f8-4282-bada-b91fa2371f12 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:23.530516 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8278870-8f5a-452b-b16a-f141becde036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:23.531140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8278870-8f5a-452b-b16a-f141becde036 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:23.531425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8278870-8f5a-452b-b16a-f141becde036 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:23.531679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8278870-8f5a-452b-b16a-f141becde036 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:23.532316 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8278870-8f5a-452b-b16a-f141becde036 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:23.532618 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8278870-8f5a-452b-b16a-f141becde036 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:23.534609 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8278870-8f5a-452b-b16a-f141becde036 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:23.547447 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 246/493] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:23.552727 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd90832-97c0-49bc-9198-3ed3e7932a00 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:23.707017 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd90832-97c0-49bc-9198-3ed3e7932a00 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:23.712122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798def65-fbd1-4146-aa9b-415d18d226df None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:23.712683 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798def65-fbd1-4146-aa9b-415d18d226df None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:23.713218 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798def65-fbd1-4146-aa9b-415d18d226df None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:23.715979 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 247/494] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:23.720180 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c55fdf-e5dc-4943-ab6b-372a1e3fca3d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:23.720715 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c55fdf-e5dc-4943-ab6b-372a1e3fca3d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:23.721868 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c55fdf-e5dc-4943-ab6b-372a1e3fca3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:23.740921 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-86c55fdf-e5dc-4943-ab6b-372a1e3fca3d None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:23.742420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-86c55fdf-e5dc-4943-ab6b-372a1e3fca3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:23.756591 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a82843-7c44-44b6-b15e-32863e29e6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:23.757262 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a82843-7c44-44b6-b15e-32863e29e6f3 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:23.757543 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a82843-7c44-44b6-b15e-32863e29e6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:23.757814 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a82843-7c44-44b6-b15e-32863e29e6f3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:23.777835 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a82843-7c44-44b6-b15e-32863e29e6f3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:23.779285 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a82843-7c44-44b6-b15e-32863e29e6f3 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:23.784408 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:23.793012 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:23.797320 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-07a82843-7c44-44b6-b15e-32863e29e6f3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 20 05:52:23.797320 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:23.797320 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:23.797320 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:23.797320 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:23.797320 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:23.797320 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:23.797320 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:23.797320 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:23.797320 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:23.797320 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:23.797320 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:23.797320 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:52:23.797320 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:52:23.797320 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:52:23.797320 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:52:23.798249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:23.798249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:23.798249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:52:23.798249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:23.798249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:52:23.798249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:23.798249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:23.798249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:23.798249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:23.798249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:23.798249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:23.798249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:23.798249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:23.798249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:23.798249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:23.798249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:23.798249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:23.798249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:23.799614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:23.799614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:23.799614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:52:23.799614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:52:23.799614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:52:23.799614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:52:23.799614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:52:23.799614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:52:23.799614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 20 05:52:23.799614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:23.799614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 248/495] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Oct 20 05:52:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:23.805065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a82843-7c44-44b6-b15e-32863e29e6f3 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:23.873377 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 248/496] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:52:23.954696 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a82843-7c44-44b6-b15e-32863e29e6f3 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:24.001865 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf7eb6-bb43-4bdd-9652-a7cc378127cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:24.002890 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf7eb6-bb43-4bdd-9652-a7cc378127cd None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:24.003390 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf7eb6-bb43-4bdd-9652-a7cc378127cd None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:24.003863 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf7eb6-bb43-4bdd-9652-a7cc378127cd None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:24.004812 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf7eb6-bb43-4bdd-9652-a7cc378127cd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:24.005339 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf7eb6-bb43-4bdd-9652-a7cc378127cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:24.007030 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf7eb6-bb43-4bdd-9652-a7cc378127cd None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:24.027705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 249/497] 10.222.0.20 () {56 vars in 1135 bytes} [Thu Oct 20 05:52:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:24.037001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c55fdf-e5dc-4943-ab6b-372a1e3fca3d None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:24.163840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c55fdf-e5dc-4943-ab6b-372a1e3fca3d None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:24.201962 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf7eb6-bb43-4bdd-9652-a7cc378127cd None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:24.213431 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e0ff5-cb11-4459-b72d-a9edf4adee40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:24.214527 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e0ff5-cb11-4459-b72d-a9edf4adee40 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:24.215009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e0ff5-cb11-4459-b72d-a9edf4adee40 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:24.215459 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e0ff5-cb11-4459-b72d-a9edf4adee40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:24.282824 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:24.284880 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:24.361549 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf7eb6-bb43-4bdd-9652-a7cc378127cd None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:24.378334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e0ff5-cb11-4459-b72d-a9edf4adee40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:24.379950 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e0ff5-cb11-4459-b72d-a9edf4adee40 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:24.409468 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b074f10-a476-4c94-8555-7c210baa5939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:24.410606 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b074f10-a476-4c94-8555-7c210baa5939 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:24.412024 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b074f10-a476-4c94-8555-7c210baa5939 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:24.412659 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b074f10-a476-4c94-8555-7c210baa5939 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:24.436290 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b074f10-a476-4c94-8555-7c210baa5939 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:52:24.444446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-0b074f10-a476-4c94-8555-7c210baa5939 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:52:24.444446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:24.444446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:24.444446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:24.444446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:24.444446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:24.444446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:24.444446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:24.444446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:24.444446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:24.444446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:24.444446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:24.444446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:52:24.444446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:52:24.444446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:52:24.444446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:52:24.445628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:52:24.445628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:52:24.445628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:52:24.445628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:52:24.445628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:24.445628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:24.445628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:24.445628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:24.445628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:24.445628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:24.445628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:24.445628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:24.445628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:24.445628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:24.445628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:24.445628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:24.446496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:24.446496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:24.446496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:24.446496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:24.446496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:52:24.446496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:52:24.446496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:52:24.446496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:52:24.446496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:52:24.446496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:52:24.446496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:52:24.446496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:24.449071 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 250/498] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Oct 20 05:52:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:52:24.457772 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b074f10-a476-4c94-8555-7c210baa5939 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:24.533994 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 249/499] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:52:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 05:52:24.535337 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d12d5-22db-4fdb-abda-0765cb67995d None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:24.535629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d12d5-22db-4fdb-abda-0765cb67995d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:24.535908 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d12d5-22db-4fdb-abda-0765cb67995d None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:24.537434 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 250/500] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:24.539263 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d12d5-22db-4fdb-abda-0765cb67995d None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:24.601168 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b074f10-a476-4c94-8555-7c210baa5939 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:24.618277 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:24.622313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:24.639194 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a1a99-8cb5-4db9-b263-549e027ea31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:24.639799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a1a99-8cb5-4db9-b263-549e027ea31b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:24.640071 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a1a99-8cb5-4db9-b263-549e027ea31b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:24.640339 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a1a99-8cb5-4db9-b263-549e027ea31b None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:24.641120 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a1a99-8cb5-4db9-b263-549e027ea31b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:24.641420 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a1a99-8cb5-4db9-b263-549e027ea31b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:24.642792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a1a99-8cb5-4db9-b263-549e027ea31b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:24.656737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 251/501] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:52:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:24.656737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1408b50-90ce-4d3a-a6b7-0b0f12d6b17a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:24.656737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1408b50-90ce-4d3a-a6b7-0b0f12d6b17a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:24.656737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1408b50-90ce-4d3a-a6b7-0b0f12d6b17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:24.673372 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c1408b50-90ce-4d3a-a6b7-0b0f12d6b17a None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:24.673984 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d12d5-22db-4fdb-abda-0765cb67995d None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:24.675252 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c1408b50-90ce-4d3a-a6b7-0b0f12d6b17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:24.721669 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ac277-795a-4cdd-9b9d-a0b676e989c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:24.722629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ac277-795a-4cdd-9b9d-a0b676e989c2 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:24.723469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ac277-795a-4cdd-9b9d-a0b676e989c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:24.724269 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ac277-795a-4cdd-9b9d-a0b676e989c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:24.740260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ac277-795a-4cdd-9b9d-a0b676e989c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:24.741760 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ac277-795a-4cdd-9b9d-a0b676e989c2 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:24.751660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 252/502] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:52:24.753609 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1408b50-90ce-4d3a-a6b7-0b0f12d6b17a None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:24.765825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-3d1ac277-795a-4cdd-9b9d-a0b676e989c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:24.765825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:24.765825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:24.765825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:24.765825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:24.765825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:24.765825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:24.765825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:24.765825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:24.765825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:24.765825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:24.765825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:24.765825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:24.765825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:24.765825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:24.765825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:24.766871 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:24.766871 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:24.766871 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:24.766871 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:24.766871 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:24.766871 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:24.766871 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:24.766871 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:24.766871 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:24.766871 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:24.766871 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:24.766871 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:24.766871 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:24.766871 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:24.766871 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:24.766871 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:24.766871 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:24.767802 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:24.767802 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:24.767802 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:24.767802 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 251/503] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:52:24.773187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ac277-795a-4cdd-9b9d-a0b676e989c2 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:24.901226 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1408b50-90ce-4d3a-a6b7-0b0f12d6b17a None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:24.919693 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ac277-795a-4cdd-9b9d-a0b676e989c2 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:24.923271 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:24.955848 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:24.973797 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b11070-50e3-49fb-baab-84e708b24099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:24.974444 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b11070-50e3-49fb-baab-84e708b24099 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:24.974723 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b11070-50e3-49fb-baab-84e708b24099 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:24.974994 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b11070-50e3-49fb-baab-84e708b24099 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:24.986397 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e2116-0b3a-49cd-8a78-d1ddf2208f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:24.987129 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e2116-0b3a-49cd-8a78-d1ddf2208f63 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:24.987346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e2116-0b3a-49cd-8a78-d1ddf2208f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:24.987621 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e2116-0b3a-49cd-8a78-d1ddf2208f63 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:24.988258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e2116-0b3a-49cd-8a78-d1ddf2208f63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:24.988560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e2116-0b3a-49cd-8a78-d1ddf2208f63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:24.990018 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e2116-0b3a-49cd-8a78-d1ddf2208f63 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:24.990941 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b11070-50e3-49fb-baab-84e708b24099 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:24.992612 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b11070-50e3-49fb-baab-84e708b24099 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:25.007869 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 252/504] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:25.013420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e2116-0b3a-49cd-8a78-d1ddf2208f63 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:25.028519 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-46b11070-50e3-49fb-baab-84e708b24099 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:25.028519 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:25.028519 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:25.028519 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:25.028519 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:25.028519 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:25.028519 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:25.028519 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:25.028519 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:25.028519 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:25.028519 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:25.028519 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:25.028519 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:25.028519 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:25.028519 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:25.028519 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:25.029649 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:25.029649 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:25.029649 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:25.029649 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:25.029649 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:25.029649 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:25.029649 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:25.029649 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:25.029649 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:25.029649 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:25.029649 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:25.029649 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:25.029649 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:25.029649 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:25.029649 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:25.029649 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:25.029649 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:25.031165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:25.031165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:25.031165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:25.031165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 253/505] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:25.036043 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b11070-50e3-49fb-baab-84e708b24099 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:25.173762 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e2116-0b3a-49cd-8a78-d1ddf2208f63 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:25.183937 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b11070-50e3-49fb-baab-84e708b24099 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:25.208628 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab58d7d-0b93-43ad-b8ea-65c5569682b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:25.209281 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab58d7d-0b93-43ad-b8ea-65c5569682b9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:25.209566 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab58d7d-0b93-43ad-b8ea-65c5569682b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:25.209835 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab58d7d-0b93-43ad-b8ea-65c5569682b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:25.216392 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab58d7d-0b93-43ad-b8ea-65c5569682b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:25.217902 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab58d7d-0b93-43ad-b8ea-65c5569682b9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:25.219543 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1cc112-4044-48bb-9a82-36327f0c7350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:25.220159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1cc112-4044-48bb-9a82-36327f0c7350 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:25.220433 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1cc112-4044-48bb-9a82-36327f0c7350 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:25.220722 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1cc112-4044-48bb-9a82-36327f0c7350 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:25.221514 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1cc112-4044-48bb-9a82-36327f0c7350 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:25.221821 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1cc112-4044-48bb-9a82-36327f0c7350 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:25.223189 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1cc112-4044-48bb-9a82-36327f0c7350 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:25.229241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-9ab58d7d-0b93-43ad-b8ea-65c5569682b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:25.229241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:25.229241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:25.229241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:25.229241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:25.229241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:25.229241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:25.229241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:25.229241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:25.229241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:25.229241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:25.229241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:25.229241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:52:25.229241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:52:25.229241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:52:25.230187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:52:25.230187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:25.230187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:25.230187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:25.230187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:25.230187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:25.230187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:25.230187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:25.230187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:25.230187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:25.230187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:25.230187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:25.230187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:25.230187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:25.230187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:25.230187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:25.230187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:25.231051 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:52:25.231051 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:52:25.231051 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:52:25.231051 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:52:25.231051 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:52:25.231051 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:52:25.231051 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:25.231051 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:25.231051 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 253/506] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:52:25.231525 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab58d7d-0b93-43ad-b8ea-65c5569682b9 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:25.243507 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 254/507] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:25.243507 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe3e35-651a-4e03-806e-79c9b999f164 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:25.243507 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe3e35-651a-4e03-806e-79c9b999f164 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:25.243507 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe3e35-651a-4e03-806e-79c9b999f164 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:25.243507 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 255/508] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:25.243507 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe3e35-651a-4e03-806e-79c9b999f164 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:25.264976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:25.285120 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:25.349297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab58d7d-0b93-43ad-b8ea-65c5569682b9 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:25.361903 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe3e35-651a-4e03-806e-79c9b999f164 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:25.385534 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfebd4ec-8e37-4ed5-be51-b342b499a662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:25.386168 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebd4ec-8e37-4ed5-be51-b342b499a662 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:25.386488 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebd4ec-8e37-4ed5-be51-b342b499a662 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:25.386715 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebd4ec-8e37-4ed5-be51-b342b499a662 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:25.387359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfebd4ec-8e37-4ed5-be51-b342b499a662 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:25.388652 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfebd4ec-8e37-4ed5-be51-b342b499a662 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:25.398714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a7b16-b00d-47cf-acf5-5b2707b90b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:25.399329 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a7b16-b00d-47cf-acf5-5b2707b90b9e None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:25.399887 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a7b16-b00d-47cf-acf5-5b2707b90b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:25.400161 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a7b16-b00d-47cf-acf5-5b2707b90b9e None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:25.400346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-dfebd4ec-8e37-4ed5-be51-b342b499a662 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 05:52:25.400346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:25.400346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:52:25.400346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:52:25.400346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:52:25.400346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:52:25.400346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:52:25.400346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:25.400346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 05:52:25.400346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:25.400346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:52:25.400346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:25.400346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:25.400346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:25.400346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:25.400346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:25.400346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:25.401286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:25.401286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:25.401286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:25.401286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:25.401286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:25.401286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:25.401286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:52:25.401286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:52:25.401286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:52:25.401286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:52:25.401286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:25.401286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:25.401286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:25.401286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:25.401286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:25.401286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:25.402123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:25.402123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:25.402123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:25.402123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:25.402123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:25.402123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:25.402123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:25.402123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:25.402123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:25.402123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:25.402123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:52:25.402123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:25.402123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 05:52:25.402123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:25.402846 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a7b16-b00d-47cf-acf5-5b2707b90b9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:25.402846 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a7b16-b00d-47cf-acf5-5b2707b90b9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:25.402971 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 254/509] 10.222.0.20 () {56 vars in 1129 bytes} [Thu Oct 20 05:52:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:25.402971 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfebd4ec-8e37-4ed5-be51-b342b499a662 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:25.407465 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a7b16-b00d-47cf-acf5-5b2707b90b9e None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:25.420147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 256/510] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Oct 20 05:52:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:25.421552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf81770-af21-455f-beae-b7d8f8436843 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:25.421843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf81770-af21-455f-beae-b7d8f8436843 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:25.422118 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf81770-af21-455f-beae-b7d8f8436843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:25.440966 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-abf81770-af21-455f-beae-b7d8f8436843 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:25.442120 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-abf81770-af21-455f-beae-b7d8f8436843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:25.512932 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 257/511] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:52:25.513851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf81770-af21-455f-beae-b7d8f8436843 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:25.523040 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfebd4ec-8e37-4ed5-be51-b342b499a662 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:25.525739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:25.531309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:25.566287 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df55e1f-6983-4f11-8c87-e32733c25deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:25.567556 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df55e1f-6983-4f11-8c87-e32733c25deb None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:25.568226 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df55e1f-6983-4f11-8c87-e32733c25deb None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:25.568642 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df55e1f-6983-4f11-8c87-e32733c25deb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:25.577696 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df55e1f-6983-4f11-8c87-e32733c25deb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:25.580239 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df55e1f-6983-4f11-8c87-e32733c25deb None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:25.600653 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-9df55e1f-6983-4f11-8c87-e32733c25deb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 20 05:52:25.600653 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:25.600653 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:25.600653 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:25.600653 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:25.600653 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:25.600653 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:25.600653 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:25.600653 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:25.600653 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:25.600653 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:25.600653 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:25.600653 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:52:25.600653 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:52:25.600653 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:52:25.600653 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:52:25.601815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:25.601815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:25.601815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:52:25.601815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:25.601815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:52:25.601815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:25.601815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:25.601815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:25.601815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:25.601815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:25.601815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:25.601815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:25.601815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:25.601815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:25.601815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:25.601815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:25.601815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:25.601815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:25.602954 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:25.602954 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:25.602954 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:52:25.602954 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:52:25.602954 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:52:25.602954 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:52:25.602954 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:52:25.602954 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:52:25.602954 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 20 05:52:25.602954 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:25.605095 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 255/512] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Oct 20 05:52:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:25.606070 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df55e1f-6983-4f11-8c87-e32733c25deb None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:25.640847 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf81770-af21-455f-beae-b7d8f8436843 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:25.690469 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b46825-0b80-4156-9e66-219dfff3093a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:25.691994 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b46825-0b80-4156-9e66-219dfff3093a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:25.692220 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b46825-0b80-4156-9e66-219dfff3093a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:25.692491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b46825-0b80-4156-9e66-219dfff3093a None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:25.693459 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b46825-0b80-4156-9e66-219dfff3093a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:25.693823 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b46825-0b80-4156-9e66-219dfff3093a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:25.695462 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b46825-0b80-4156-9e66-219dfff3093a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:25.701671 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 258/513] 10.222.0.20 () {56 vars in 1124 bytes} [Thu Oct 20 05:52:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:25.702477 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b46825-0b80-4156-9e66-219dfff3093a None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:25.755877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df55e1f-6983-4f11-8c87-e32733c25deb None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:25.793773 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:25.798191 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0cfe3e-cecd-404d-9fca-9423ced68732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:25.798953 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cfe3e-cecd-404d-9fca-9423ced68732 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:25.800015 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cfe3e-cecd-404d-9fca-9423ced68732 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:25.800293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cfe3e-cecd-404d-9fca-9423ced68732 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:25.801214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0cfe3e-cecd-404d-9fca-9423ced68732 None admin] RBAC: Adding query filter params (role.id=afd859d2deed4d51aa3f6326920441c8, scope.project.id=0ee65b9a8229495bbe97318f060d6342, user.id=eeeab506ab5c4d6386fc899086f0abf5) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:25.801525 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0cfe3e-cecd-404d-9fca-9423ced68732 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:25.803358 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0cfe3e-cecd-404d-9fca-9423ced68732 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:25.812821 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 256/514] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:52:25 2022] GET /identity/v3/role_assignments?user.id=eeeab506ab5c4d6386fc899086f0abf5&scope.project.id=0ee65b9a8229495bbe97318f060d6342&role.id=afd859d2deed4d51aa3f6326920441c8 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:25.812821 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0cfe3e-cecd-404d-9fca-9423ced68732 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:25.824304 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b46825-0b80-4156-9e66-219dfff3093a None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:25.851747 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:25.863582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c45d7-af50-4608-a3f8-8706ca08d19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:25.864219 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c45d7-af50-4608-a3f8-8706ca08d19e None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:25.864500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c45d7-af50-4608-a3f8-8706ca08d19e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:25.864814 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c45d7-af50-4608-a3f8-8706ca08d19e None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:25.865561 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c45d7-af50-4608-a3f8-8706ca08d19e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:25.865879 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c45d7-af50-4608-a3f8-8706ca08d19e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:25.867366 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c45d7-af50-4608-a3f8-8706ca08d19e None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:25.891513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 259/515] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Oct 20 05:52:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:25.891513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30e34c-a662-43f7-8608-bcb787be4633 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:25.891513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30e34c-a662-43f7-8608-bcb787be4633 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:25.891513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30e34c-a662-43f7-8608-bcb787be4633 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:25.891513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 260/516] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:25.891513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30e34c-a662-43f7-8608-bcb787be4633 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:25.945086 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0cfe3e-cecd-404d-9fca-9423ced68732 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:25.986750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486fced-81d8-428f-881c-fc8213cabd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:25.988539 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486fced-81d8-428f-881c-fc8213cabd1b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:25.988539 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486fced-81d8-428f-881c-fc8213cabd1b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:25.988539 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486fced-81d8-428f-881c-fc8213cabd1b None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:25.988817 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7486fced-81d8-428f-881c-fc8213cabd1b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:25.988994 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7486fced-81d8-428f-881c-fc8213cabd1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:25.990351 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7486fced-81d8-428f-881c-fc8213cabd1b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:25.996304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 257/517] 10.222.0.20 () {56 vars in 1124 bytes} [Thu Oct 20 05:52:25 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:25.998052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486fced-81d8-428f-881c-fc8213cabd1b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:26.007377 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30e34c-a662-43f7-8608-bcb787be4633 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:26.050210 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:26.056640 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc7b08-7a77-40a6-9c03-16ee78e87408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:26.057328 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc7b08-7a77-40a6-9c03-16ee78e87408 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:26.057603 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc7b08-7a77-40a6-9c03-16ee78e87408 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:26.057867 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc7b08-7a77-40a6-9c03-16ee78e87408 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:26.072352 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc7b08-7a77-40a6-9c03-16ee78e87408 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:26.073502 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc7b08-7a77-40a6-9c03-16ee78e87408 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:26.090306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-30bc7b08-7a77-40a6-9c03-16ee78e87408 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:26.090306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:26.090306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:26.090306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:26.090306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:26.090306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:26.090306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:26.090306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:26.090306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:26.090306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:26.090306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:26.090306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:26.090306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:26.090306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:26.090306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:26.090306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:26.091332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:26.091332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:26.091332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:26.091332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:26.091332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:26.091332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:26.091332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:26.091332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:26.091332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:26.091332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:26.091332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:26.091332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:26.091332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:26.091332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:26.091332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:26.091332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:26.091332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:26.092256 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:26.092256 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:26.092256 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:26.092256 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 261/518] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:26.093030 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedc817-7620-4ef9-8077-14691b101ca7 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:26.093312 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedc817-7620-4ef9-8077-14691b101ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:26.093578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedc817-7620-4ef9-8077-14691b101ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:26.113197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-9fedc817-7620-4ef9-8077-14691b101ca7 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:26.114419 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-9fedc817-7620-4ef9-8077-14691b101ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:26.123059 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:26.130094 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486fced-81d8-428f-881c-fc8213cabd1b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:26.172437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568609e-cb1f-4c71-a540-929635208bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:26.175362 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568609e-cb1f-4c71-a540-929635208bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:26.175657 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568609e-cb1f-4c71-a540-929635208bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:26.175935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568609e-cb1f-4c71-a540-929635208bc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:26.187309 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568609e-cb1f-4c71-a540-929635208bc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:26.188795 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568609e-cb1f-4c71-a540-929635208bc2 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:26.189657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 262/519] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:52:26.191344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fedc817-7620-4ef9-8077-14691b101ca7 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:26.207745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-b568609e-cb1f-4c71-a540-929635208bc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:26.207745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:26.207745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:26.207745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:26.207745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:26.207745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:26.207745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:26.207745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:26.207745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:26.207745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:26.207745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:26.207745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:26.207745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:26.207745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:26.207745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:26.207745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:26.208803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:26.208803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:26.208803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:26.208803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:26.208803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:26.208803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:26.208803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:26.208803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:26.208803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:26.208803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:26.208803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:26.208803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:26.208803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:26.208803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:26.208803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:26.208803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:26.208803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:26.210034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:26.210034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:26.210034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:26.210034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 258/520] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:26.214506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568609e-cb1f-4c71-a540-929635208bc2 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:26.330994 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568609e-cb1f-4c71-a540-929635208bc2 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:26.337142 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fedc817-7620-4ef9-8077-14691b101ca7 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:26.370796 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645b17b-24e8-47fe-a950-9bcd6f8df31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:26.371415 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645b17b-24e8-47fe-a950-9bcd6f8df31e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:26.371694 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645b17b-24e8-47fe-a950-9bcd6f8df31e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:26.371969 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645b17b-24e8-47fe-a950-9bcd6f8df31e None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:26.372608 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645b17b-24e8-47fe-a950-9bcd6f8df31e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:26.372941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645b17b-24e8-47fe-a950-9bcd6f8df31e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:26.374329 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645b17b-24e8-47fe-a950-9bcd6f8df31e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:26.377041 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:26.403528 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 259/521] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:26.410143 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645b17b-24e8-47fe-a950-9bcd6f8df31e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:26.419437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:26.426020 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22319d2-f7b9-413d-a8d5-4418b5e5ad79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:26.426654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22319d2-f7b9-413d-a8d5-4418b5e5ad79 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:26.426957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22319d2-f7b9-413d-a8d5-4418b5e5ad79 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:26.427732 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22319d2-f7b9-413d-a8d5-4418b5e5ad79 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:26.428458 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22319d2-f7b9-413d-a8d5-4418b5e5ad79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:26.429190 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22319d2-f7b9-413d-a8d5-4418b5e5ad79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:26.431120 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22319d2-f7b9-413d-a8d5-4418b5e5ad79 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:26.456833 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 263/522] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:26.459130 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22319d2-f7b9-413d-a8d5-4418b5e5ad79 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:26.587161 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645b17b-24e8-47fe-a950-9bcd6f8df31e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:26.605584 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22319d2-f7b9-413d-a8d5-4418b5e5ad79 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:26.633342 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe8032-da70-4414-90f2-05c67fd53a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:26.634594 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe8032-da70-4414-90f2-05c67fd53a09 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:26.635473 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe8032-da70-4414-90f2-05c67fd53a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:26.636239 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe8032-da70-4414-90f2-05c67fd53a09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:26.646390 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf431cc4-23d2-4746-8a77-114b7b0ff479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:26.647010 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf431cc4-23d2-4746-8a77-114b7b0ff479 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:26.647291 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf431cc4-23d2-4746-8a77-114b7b0ff479 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:26.647565 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf431cc4-23d2-4746-8a77-114b7b0ff479 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:26.648228 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf431cc4-23d2-4746-8a77-114b7b0ff479 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:26.649449 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fe8032-da70-4414-90f2-05c67fd53a09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:26.649694 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf431cc4-23d2-4746-8a77-114b7b0ff479 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:26.651327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fe8032-da70-4414-90f2-05c67fd53a09 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:26.666620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-05fe8032-da70-4414-90f2-05c67fd53a09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:26.666620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:26.666620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:26.666620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:26.666620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:26.666620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:26.666620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:26.666620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:26.666620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:26.666620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:26.666620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:26.666620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:26.666620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:52:26.666620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:52:26.666620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:52:26.667626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:52:26.667626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:26.667626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:26.667626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:26.667626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:26.667626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:26.667626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:26.667626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:26.667626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:26.667626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:26.667626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:26.667626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:26.667626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:26.667626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:26.667626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:26.667626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:26.667626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:26.668679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:52:26.668679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:52:26.668679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:52:26.668679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:52:26.668679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:52:26.668679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:52:26.668679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:26.668679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:26.676059 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 260/523] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:26.676059 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe8032-da70-4414-90f2-05c67fd53a09 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:26.685560 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 264/524] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Oct 20 05:52:26 2022] POST /identity/v3/users => generated 254 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:52:26.687290 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf431cc4-23d2-4746-8a77-114b7b0ff479 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:26.724592 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:26.771620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:26.822017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe8032-da70-4414-90f2-05c67fd53a09 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:26.864547 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8fcff-1f6d-4690-9f97-262a228ed051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:26.865734 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8fcff-1f6d-4690-9f97-262a228ed051 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:26.866235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8fcff-1f6d-4690-9f97-262a228ed051 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:26.866717 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8fcff-1f6d-4690-9f97-262a228ed051 None admin] PATH_INFO: `/v3/services/network` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:26.867521 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a8fcff-1f6d-4690-9f97-262a228ed051 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:26.869304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a8fcff-1f6d-4690-9f97-262a228ed051 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:26.880205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-85a8fcff-1f6d-4690-9f97-262a228ed051 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 05:52:26.880205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:26.880205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:52:26.880205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:52:26.880205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:52:26.880205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:52:26.880205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:52:26.880205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:26.880205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 05:52:26.880205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:26.880205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:52:26.880205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:26.880205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:26.880205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:26.880205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:26.880205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:26.880205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:26.881345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:26.881345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:26.881345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:26.881345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:26.881345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:26.881345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:26.881345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:52:26.881345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:52:26.881345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:52:26.881345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:52:26.881345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:26.881345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:26.881345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:26.881345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:26.881345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:26.881345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:26.882251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:26.882251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:26.882251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:26.882251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:26.882251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:26.882251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:26.882251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:26.882251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:26.882251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:26.882251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:26.882251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:52:26.882251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:26.882251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 05:52:26.882251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:26.883100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf431cc4-23d2-4746-8a77-114b7b0ff479 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:26.885729 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 261/525] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:26 2022] GET /identity/v3/services/network => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:26.892547 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8fcff-1f6d-4690-9f97-262a228ed051 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:26.949629 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c33b75-dbd7-4074-90f0-a926b25bdce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:26.950282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c33b75-dbd7-4074-90f0-a926b25bdce5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:26.950607 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c33b75-dbd7-4074-90f0-a926b25bdce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:26.950879 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c33b75-dbd7-4074-90f0-a926b25bdce5 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:26.951569 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c33b75-dbd7-4074-90f0-a926b25bdce5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:26.951870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c33b75-dbd7-4074-90f0-a926b25bdce5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:26.953347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c33b75-dbd7-4074-90f0-a926b25bdce5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:26.978992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 265/526] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Oct 20 05:52:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:26.985069 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c33b75-dbd7-4074-90f0-a926b25bdce5 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:27.021547 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8fcff-1f6d-4690-9f97-262a228ed051 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:27.058295 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:27.070511 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338cc6f9-abaa-418b-be38-a639a7ca13a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:27.072023 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338cc6f9-abaa-418b-be38-a639a7ca13a5 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:27.072486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338cc6f9-abaa-418b-be38-a639a7ca13a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:27.072974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338cc6f9-abaa-418b-be38-a639a7ca13a5 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:27.073771 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338cc6f9-abaa-418b-be38-a639a7ca13a5 None admin] RBAC: Adding query filter params (name=network) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:27.074257 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338cc6f9-abaa-418b-be38-a639a7ca13a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:27.076644 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338cc6f9-abaa-418b-be38-a639a7ca13a5 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:27.083656 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 262/527] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Oct 20 05:52:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:27.090085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338cc6f9-abaa-418b-be38-a639a7ca13a5 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:27.110930 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:27.125064 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c33b75-dbd7-4074-90f0-a926b25bdce5 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:27.169103 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db376129-3676-4779-8bc8-27a603bf89f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:27.169871 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db376129-3676-4779-8bc8-27a603bf89f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:27.170214 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db376129-3676-4779-8bc8-27a603bf89f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:27.170560 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db376129-3676-4779-8bc8-27a603bf89f9 None admin] PATH_INFO: `/v3/projects/0ee65b9a8229495bbe97318f060d6342/users/ae2c1c886cd94d9f90082f66e7e8c9f4/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:27.172988 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db376129-3676-4779-8bc8-27a603bf89f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ee65b9a8229495bbe97318f060d6342, user_id=ae2c1c886cd94d9f90082f66e7e8c9f4, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:27.174355 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db376129-3676-4779-8bc8-27a603bf89f9 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:27.196834 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 266/528] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:52:26 2022] PUT /identity/v3/projects/0ee65b9a8229495bbe97318f060d6342/users/ae2c1c886cd94d9f90082f66e7e8c9f4/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 05:52:27.235335 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338cc6f9-abaa-418b-be38-a639a7ca13a5 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:27.281484 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe6b4a9-0548-4360-9ea0-92ed52ff66e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:27.282425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6b4a9-0548-4360-9ea0-92ed52ff66e4 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:27.282902 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6b4a9-0548-4360-9ea0-92ed52ff66e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:27.283358 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6b4a9-0548-4360-9ea0-92ed52ff66e4 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:27.284767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe6b4a9-0548-4360-9ea0-92ed52ff66e4 None admin] RBAC: Adding query filter params (type=network) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:27.285253 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe6b4a9-0548-4360-9ea0-92ed52ff66e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:27.286779 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe6b4a9-0548-4360-9ea0-92ed52ff66e4 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:27.296763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 263/529] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Oct 20 05:52:27 2022] GET /identity/v3/services?type=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:28.186206 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cefdc3-e6f4-4b87-ae22-9e08f1b4f5b6 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:28.187116 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cefdc3-e6f4-4b87-ae22-9e08f1b4f5b6 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:28.187581 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cefdc3-e6f4-4b87-ae22-9e08f1b4f5b6 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:28.189438 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 267/530] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:28.196670 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ed006-29b6-4762-bbab-5527c686738b None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:28.197623 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ed006-29b6-4762-bbab-5527c686738b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:28.198068 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ed006-29b6-4762-bbab-5527c686738b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:28.221688 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-904ed006-29b6-4762-bbab-5527c686738b None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:28.223166 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-904ed006-29b6-4762-bbab-5527c686738b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:28.289324 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:28.308797 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 264/531] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:52:28.480872 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794adccd-76ea-40fa-9e72-a87ea487b122 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:28.481157 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794adccd-76ea-40fa-9e72-a87ea487b122 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:28.481432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794adccd-76ea-40fa-9e72-a87ea487b122 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:28.482990 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 268/532] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:28.494156 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7223f3-86f4-4eb6-a205-1cf673bee607 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:28.494460 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7223f3-86f4-4eb6-a205-1cf673bee607 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:28.494708 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7223f3-86f4-4eb6-a205-1cf673bee607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:28.518794 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-fc7223f3-86f4-4eb6-a205-1cf673bee607 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:28.520297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-fc7223f3-86f4-4eb6-a205-1cf673bee607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:28.609330 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 265/533] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:52:28.671910 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794adccd-76ea-40fa-9e72-a87ea487b122 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:28.809727 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794adccd-76ea-40fa-9e72-a87ea487b122 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:28.850819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d1b5d-faa2-4e43-87de-1e6b71c6d07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:28.851451 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d1b5d-faa2-4e43-87de-1e6b71c6d07d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:28.851727 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d1b5d-faa2-4e43-87de-1e6b71c6d07d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:28.851994 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d1b5d-faa2-4e43-87de-1e6b71c6d07d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:28.869491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d1b5d-faa2-4e43-87de-1e6b71c6d07d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:52:28.874236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-3d2d1b5d-faa2-4e43-87de-1e6b71c6d07d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:52:28.874236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:28.874236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:28.874236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:28.874236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:28.874236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:28.874236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:28.874236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:28.874236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:28.874236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:28.874236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:28.874236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:28.874236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:52:28.874236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:52:28.874236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:52:28.874236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:52:28.875165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:52:28.875165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:52:28.875165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:52:28.875165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:52:28.875165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:28.875165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:28.875165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:28.875165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:28.875165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:28.875165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:28.875165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:28.875165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:28.875165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:28.875165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:28.875165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:28.875165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:28.876533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:28.876533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:28.876533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:28.876533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:28.876533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:52:28.876533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:52:28.876533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:52:28.876533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:52:28.876533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:52:28.876533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:52:28.876533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:52:28.876533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:28.876533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 269/534] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Oct 20 05:52:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:28.881950 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7223f3-86f4-4eb6-a205-1cf673bee607 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:29.009769 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d1b5d-faa2-4e43-87de-1e6b71c6d07d None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:29.038173 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:29.044188 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7223f3-86f4-4eb6-a205-1cf673bee607 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:29.065049 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:29.107492 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d6c20-6cd9-474a-a960-11cb5813f5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:29.108396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d6c20-6cd9-474a-a960-11cb5813f5ae None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:29.108859 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d6c20-6cd9-474a-a960-11cb5813f5ae None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:29.109290 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d6c20-6cd9-474a-a960-11cb5813f5ae None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:29.110099 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d6c20-6cd9-474a-a960-11cb5813f5ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:29.110574 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d6c20-6cd9-474a-a960-11cb5813f5ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:29.113613 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d6c20-6cd9-474a-a960-11cb5813f5ae None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:29.133067 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 266/535] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:52:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:29.138795 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d6c20-6cd9-474a-a960-11cb5813f5ae None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:29.203317 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d1b5d-faa2-4e43-87de-1e6b71c6d07d None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:29.270229 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e86f05-a700-40dc-8411-01deb2de3f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:29.270843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e86f05-a700-40dc-8411-01deb2de3f8d None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:29.271118 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e86f05-a700-40dc-8411-01deb2de3f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:29.271386 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e86f05-a700-40dc-8411-01deb2de3f8d None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:29.272052 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e86f05-a700-40dc-8411-01deb2de3f8d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:29.278166 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e86f05-a700-40dc-8411-01deb2de3f8d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:29.316895 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 270/536] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Oct 20 05:52:29 2022] POST /identity/v3/services => generated 251 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 05:52:29.331909 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d6c20-6cd9-474a-a960-11cb5813f5ae None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:29.378392 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b7e9b-e017-422c-af58-7540e01ba476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:29.379050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b7e9b-e017-422c-af58-7540e01ba476 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:29.379326 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b7e9b-e017-422c-af58-7540e01ba476 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:29.379602 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b7e9b-e017-422c-af58-7540e01ba476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:29.391979 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b7e9b-e017-422c-af58-7540e01ba476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:29.394154 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b7e9b-e017-422c-af58-7540e01ba476 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:29.410537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-bf1b7e9b-e017-422c-af58-7540e01ba476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:29.410537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:29.410537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:29.410537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:29.410537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:29.410537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:29.410537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:29.410537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:29.410537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:29.410537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:29.410537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:29.410537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:29.410537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:29.410537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:29.410537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:29.410537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:29.411552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:29.411552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:29.411552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:29.411552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:29.411552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:29.411552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:29.411552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:29.411552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:29.411552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:29.411552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:29.411552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:29.411552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:29.411552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:29.411552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:29.411552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:29.411552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:29.411552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:29.413108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:29.413108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:29.413108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:29.413108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 267/537] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:52:29.417326 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e86f05-a700-40dc-8411-01deb2de3f8d None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:29.495202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:29.543148 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e86f05-a700-40dc-8411-01deb2de3f8d None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:29.598039 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8302b32-887d-45e6-8a94-d6887b883e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:29.598595 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8302b32-887d-45e6-8a94-d6887b883e9b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:29.598877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8302b32-887d-45e6-8a94-d6887b883e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:29.599149 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8302b32-887d-45e6-8a94-d6887b883e9b None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:29.599804 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8302b32-887d-45e6-8a94-d6887b883e9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:29.600107 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8302b32-887d-45e6-8a94-d6887b883e9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:29.601512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8302b32-887d-45e6-8a94-d6887b883e9b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:29.619249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 271/538] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:29.624549 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b7e9b-e017-422c-af58-7540e01ba476 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:29.656602 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:29.747355 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b7e9b-e017-422c-af58-7540e01ba476 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:29.786714 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72f277-940a-4201-b27b-278523c3221d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:29.787623 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72f277-940a-4201-b27b-278523c3221d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:29.788045 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72f277-940a-4201-b27b-278523c3221d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:29.788531 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72f277-940a-4201-b27b-278523c3221d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:29.796912 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72f277-940a-4201-b27b-278523c3221d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:29.798698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72f277-940a-4201-b27b-278523c3221d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:29.816293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-5b72f277-940a-4201-b27b-278523c3221d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 20 05:52:29.816293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:29.816293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:29.816293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:29.816293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:29.816293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:29.816293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:29.816293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:29.816293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:29.816293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:29.816293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:29.816293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:29.816293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:52:29.816293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:52:29.816293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:52:29.816293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:52:29.817495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:29.817495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:29.817495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:52:29.817495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:29.817495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:52:29.817495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:29.817495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:29.817495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:29.817495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:29.817495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:29.817495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:29.817495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:29.817495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:29.817495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:29.817495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:29.817495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:29.817495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:29.817495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:29.818450 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:29.818450 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:29.818450 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:52:29.818450 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:52:29.818450 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:52:29.818450 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:52:29.818450 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:52:29.818450 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:52:29.818450 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 20 05:52:29.818450 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:29.825171 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 268/539] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Oct 20 05:52:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:29.825381 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467002a-8cb4-411c-ac29-b3ebb91e8a57 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:29.826039 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467002a-8cb4-411c-ac29-b3ebb91e8a57 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:29.826039 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467002a-8cb4-411c-ac29-b3ebb91e8a57 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:29.829109 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 272/540] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:29.831164 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72f277-940a-4201-b27b-278523c3221d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:29.834490 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bd2ce-d7e0-4932-96eb-b9be279dfac2 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:29.835111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bd2ce-d7e0-4932-96eb-b9be279dfac2 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:29.835470 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bd2ce-d7e0-4932-96eb-b9be279dfac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:29.870635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-8f8bd2ce-d7e0-4932-96eb-b9be279dfac2 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:29.873204 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8f8bd2ce-d7e0-4932-96eb-b9be279dfac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:29.970962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:29.988080 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 273/541] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:29 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 05:52:29.989097 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72f277-940a-4201-b27b-278523c3221d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:29.989364 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895db93-7243-424e-9b1e-02a59b2227a4 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:29.989646 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895db93-7243-424e-9b1e-02a59b2227a4 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:29.989913 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895db93-7243-424e-9b1e-02a59b2227a4 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:29.992760 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 274/542] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:29.996638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9cdd8-d9ff-4a15-bd98-f92f055a0488 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:29.996949 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9cdd8-d9ff-4a15-bd98-f92f055a0488 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:29.997218 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9cdd8-d9ff-4a15-bd98-f92f055a0488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:30.021566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-fbe9cdd8-d9ff-4a15-bd98-f92f055a0488 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:30.023812 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fbe9cdd8-d9ff-4a15-bd98-f92f055a0488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:30.043035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c34eda-9b7c-471a-be5b-557f7cfbc5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:30.043646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c34eda-9b7c-471a-be5b-557f7cfbc5b8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:30.043924 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c34eda-9b7c-471a-be5b-557f7cfbc5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:30.044188 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c34eda-9b7c-471a-be5b-557f7cfbc5b8 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:30.044939 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c34eda-9b7c-471a-be5b-557f7cfbc5b8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:30.045244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c34eda-9b7c-471a-be5b-557f7cfbc5b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:30.046664 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c34eda-9b7c-471a-be5b-557f7cfbc5b8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:30.082593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 269/543] 10.222.0.20 () {56 vars in 1135 bytes} [Thu Oct 20 05:52:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:30.087691 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c34eda-9b7c-471a-be5b-557f7cfbc5b8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:30.119277 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:30.141556 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 275/544] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:29 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 05:52:30.226254 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275593d-baf9-40b5-bddf-79c9d330c70b None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:30.226895 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275593d-baf9-40b5-bddf-79c9d330c70b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:30.227853 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275593d-baf9-40b5-bddf-79c9d330c70b None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:30.229536 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 276/545] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:30.234838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741397f-36ad-4047-a757-a2d61ffa284c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:30.235117 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741397f-36ad-4047-a757-a2d61ffa284c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:30.235391 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741397f-36ad-4047-a757-a2d61ffa284c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:30.236856 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c34eda-9b7c-471a-be5b-557f7cfbc5b8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:30.258703 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-2741397f-36ad-4047-a757-a2d61ffa284c None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:30.260145 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2741397f-36ad-4047-a757-a2d61ffa284c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:30.296433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173d73d-16ea-4b01-8f6a-3ea9a6d9d685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:30.297093 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173d73d-16ea-4b01-8f6a-3ea9a6d9d685 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:30.297376 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173d73d-16ea-4b01-8f6a-3ea9a6d9d685 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:30.297644 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173d73d-16ea-4b01-8f6a-3ea9a6d9d685 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:30.318175 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173d73d-16ea-4b01-8f6a-3ea9a6d9d685 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:30.320266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173d73d-16ea-4b01-8f6a-3ea9a6d9d685 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:30.324409 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:30.341629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-8173d73d-16ea-4b01-8f6a-3ea9a6d9d685 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:30.341629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:30.341629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:30.341629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:30.341629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:30.341629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:30.341629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:30.341629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:30.341629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:30.341629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:30.341629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:30.341629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:30.341629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:30.341629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:30.341629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:30.341629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:30.343053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:30.343053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:30.343053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:30.343053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:30.343053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:30.343053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:30.343053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:30.343053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:30.343053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:30.343053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:30.343053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:30.343053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:30.343053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:30.343053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:30.343053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:30.343053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:30.343053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:30.343967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:30.343967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:30.343967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:30.351056 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 270/546] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:52:30.351056 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173d73d-16ea-4b01-8f6a-3ea9a6d9d685 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:30.368289 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 277/547] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 05:52:30.369653 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741397f-36ad-4047-a757-a2d61ffa284c None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:30.478850 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741397f-36ad-4047-a757-a2d61ffa284c None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:30.506085 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:30.525038 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c5ca8-e40b-49e5-9ddb-055fa5db1df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:30.526002 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c5ca8-e40b-49e5-9ddb-055fa5db1df6 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:30.526509 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c5ca8-e40b-49e5-9ddb-055fa5db1df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:30.526697 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173d73d-16ea-4b01-8f6a-3ea9a6d9d685 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:30.527182 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c5ca8-e40b-49e5-9ddb-055fa5db1df6 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:30.528027 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c5ca8-e40b-49e5-9ddb-055fa5db1df6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:30.528538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c5ca8-e40b-49e5-9ddb-055fa5db1df6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:30.530493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c5ca8-e40b-49e5-9ddb-055fa5db1df6 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:30.547406 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 278/548] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:30.548608 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c5ca8-e40b-49e5-9ddb-055fa5db1df6 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:30.568019 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e60be6-f00e-4ce1-858f-f92958dca49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:30.568722 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e60be6-f00e-4ce1-858f-f92958dca49f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:30.569008 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e60be6-f00e-4ce1-858f-f92958dca49f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:30.569495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e60be6-f00e-4ce1-858f-f92958dca49f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:30.583395 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e60be6-f00e-4ce1-858f-f92958dca49f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:52:30.590913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-d7e60be6-f00e-4ce1-858f-f92958dca49f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:52:30.590913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:30.590913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:30.590913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:30.590913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:30.590913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:30.590913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:30.590913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:30.590913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:30.590913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:30.590913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:30.590913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:30.590913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:52:30.590913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:52:30.590913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:52:30.590913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:52:30.591862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:52:30.591862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:52:30.591862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:52:30.591862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:52:30.591862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:30.591862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:30.591862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:30.591862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:30.591862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:30.591862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:30.591862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:30.591862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:30.591862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:30.591862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:30.591862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:30.591862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:30.593104 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:30.593104 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:30.593104 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:30.593104 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:30.593104 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:52:30.593104 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:52:30.593104 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:52:30.593104 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:52:30.593104 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:52:30.593104 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:52:30.593104 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:52:30.593104 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:30.593104 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 271/549] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Oct 20 05:52:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:30.593104 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e60be6-f00e-4ce1-858f-f92958dca49f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:30.666321 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c5ca8-e40b-49e5-9ddb-055fa5db1df6 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:30.669627 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:30.713086 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e39d9d-0277-42d4-9d8e-34473d5baaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:30.713827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e39d9d-0277-42d4-9d8e-34473d5baaa6 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:30.714101 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e39d9d-0277-42d4-9d8e-34473d5baaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:30.714369 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e39d9d-0277-42d4-9d8e-34473d5baaa6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:30.725550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e39d9d-0277-42d4-9d8e-34473d5baaa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:52:30.726257 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e60be6-f00e-4ce1-858f-f92958dca49f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:30.730838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-a6e39d9d-0277-42d4-9d8e-34473d5baaa6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:52:30.730838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:30.730838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:30.730838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:30.730838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:30.730838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:30.730838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:30.730838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:30.730838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:30.730838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:30.730838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:30.730838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:30.730838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:52:30.730838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:52:30.730838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:52:30.730838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:52:30.731745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:52:30.731745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:52:30.731745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:52:30.731745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:52:30.731745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:30.731745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:30.731745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:30.731745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:30.731745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:30.731745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:30.731745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:30.731745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:30.731745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:30.731745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:30.731745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:30.731745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:30.733274 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:30.733274 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:30.733274 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:30.733274 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:30.733274 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:52:30.733274 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:52:30.733274 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:52:30.733274 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:52:30.733274 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:52:30.733274 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:52:30.733274 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:52:30.733274 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:30.733274 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 279/550] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Oct 20 05:52:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:30.733274 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e39d9d-0277-42d4-9d8e-34473d5baaa6 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:30.767275 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc92f14-396b-4956-94d5-6520d53d1df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:30.768609 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc92f14-396b-4956-94d5-6520d53d1df8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:30.769145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc92f14-396b-4956-94d5-6520d53d1df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:30.769521 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc92f14-396b-4956-94d5-6520d53d1df8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:30.776125 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc92f14-396b-4956-94d5-6520d53d1df8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:30.776634 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:30.777726 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc92f14-396b-4956-94d5-6520d53d1df8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:30.790114 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-8bc92f14-396b-4956-94d5-6520d53d1df8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:30.790114 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:30.790114 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:30.790114 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:30.790114 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:30.790114 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:30.790114 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:30.790114 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:30.790114 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:30.790114 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:30.790114 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:30.790114 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:30.790114 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:52:30.790114 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:52:30.790114 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:52:30.791050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:52:30.791050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:30.791050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:30.791050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:30.791050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:30.791050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:30.791050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:30.791050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:30.791050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:30.791050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:30.791050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:30.791050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:30.791050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:30.791050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:30.791050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:30.791050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:30.791050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:30.791965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:52:30.791965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:52:30.791965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:52:30.791965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:52:30.791965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:52:30.791965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:52:30.791965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:30.791965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:30.791965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 272/551] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:30.791965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc92f14-396b-4956-94d5-6520d53d1df8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:30.841993 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e39d9d-0277-42d4-9d8e-34473d5baaa6 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:30.879169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b33cff-d67a-4b8d-9604-4a4394c9f3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:30.879808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b33cff-d67a-4b8d-9604-4a4394c9f3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:30.880084 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b33cff-d67a-4b8d-9604-4a4394c9f3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:30.880351 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b33cff-d67a-4b8d-9604-4a4394c9f3c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:30.881029 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b33cff-d67a-4b8d-9604-4a4394c9f3c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:30.881328 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b33cff-d67a-4b8d-9604-4a4394c9f3c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:30.882680 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b33cff-d67a-4b8d-9604-4a4394c9f3c0 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:30.891487 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 280/552] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Oct 20 05:52:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:30.892149 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b33cff-d67a-4b8d-9604-4a4394c9f3c0 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:30.901870 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc92f14-396b-4956-94d5-6520d53d1df8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:30.919197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:30.939340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece834c6-534b-40c0-90b4-969dedd6a024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:30.939982 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece834c6-534b-40c0-90b4-969dedd6a024 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:30.940312 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece834c6-534b-40c0-90b4-969dedd6a024 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:30.940592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece834c6-534b-40c0-90b4-969dedd6a024 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:30.941313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece834c6-534b-40c0-90b4-969dedd6a024 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:30.942692 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece834c6-534b-40c0-90b4-969dedd6a024 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:30.960742 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 273/553] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Oct 20 05:52:30 2022] POST /identity/v3/services => generated 240 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:52:30.960742 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece834c6-534b-40c0-90b4-969dedd6a024 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:31.021073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b33cff-d67a-4b8d-9604-4a4394c9f3c0 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:31.022790 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:31.083371 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbde004-424f-48fb-8130-46b180983ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:31.084632 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbde004-424f-48fb-8130-46b180983ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:31.085126 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbde004-424f-48fb-8130-46b180983ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:31.085538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbde004-424f-48fb-8130-46b180983ce1 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:31.086387 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbde004-424f-48fb-8130-46b180983ce1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:31.086848 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbde004-424f-48fb-8130-46b180983ce1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:31.088987 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbde004-424f-48fb-8130-46b180983ce1 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:31.104627 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece834c6-534b-40c0-90b4-969dedd6a024 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:31.105390 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 281/554] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:52:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:31.107540 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbde004-424f-48fb-8130-46b180983ce1 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:31.142108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725d627-9d2e-4c2f-b514-3da02ecfd9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:31.142812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725d627-9d2e-4c2f-b514-3da02ecfd9df None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:31.143642 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725d627-9d2e-4c2f-b514-3da02ecfd9df None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:31.143913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725d627-9d2e-4c2f-b514-3da02ecfd9df None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:31.145564 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725d627-9d2e-4c2f-b514-3da02ecfd9df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:31.145897 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725d627-9d2e-4c2f-b514-3da02ecfd9df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:31.147326 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725d627-9d2e-4c2f-b514-3da02ecfd9df None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:31.162462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 274/555] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Oct 20 05:52:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:31.164406 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725d627-9d2e-4c2f-b514-3da02ecfd9df None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:31.206382 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:31.231138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbde004-424f-48fb-8130-46b180983ce1 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:31.269748 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a014d-495f-4501-97f1-541d7c10217f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:31.270755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a014d-495f-4501-97f1-541d7c10217f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:31.271291 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a014d-495f-4501-97f1-541d7c10217f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:31.271786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a014d-495f-4501-97f1-541d7c10217f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:31.283115 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725d627-9d2e-4c2f-b514-3da02ecfd9df None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:31.283564 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a014d-495f-4501-97f1-541d7c10217f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:31.285348 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a014d-495f-4501-97f1-541d7c10217f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:31.303546 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-1a0a014d-495f-4501-97f1-541d7c10217f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:31.303546 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:31.303546 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:31.303546 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:31.303546 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:31.303546 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:31.303546 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:31.303546 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:31.303546 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:31.303546 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:31.303546 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:31.303546 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:31.303546 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:31.303546 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:31.303546 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:31.303546 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:31.304598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:31.304598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:31.304598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:31.304598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:31.304598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:31.304598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:31.304598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:31.304598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:31.304598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:31.304598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:31.304598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:31.304598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:31.304598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:31.304598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:31.304598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:31.304598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:31.304598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:31.305545 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:31.305545 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:31.305545 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:31.307078 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 282/556] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:31.308168 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a014d-495f-4501-97f1-541d7c10217f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:31.324451 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:31.328899 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767fe7d8-cfd8-40cb-84df-d6161d859876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:31.330953 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fe7d8-cfd8-40cb-84df-d6161d859876 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:31.331753 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fe7d8-cfd8-40cb-84df-d6161d859876 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:31.332200 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fe7d8-cfd8-40cb-84df-d6161d859876 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:31.346382 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767fe7d8-cfd8-40cb-84df-d6161d859876 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:31.348811 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767fe7d8-cfd8-40cb-84df-d6161d859876 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:31.370551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-767fe7d8-cfd8-40cb-84df-d6161d859876 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:31.370551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:31.370551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:31.370551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:31.370551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:31.370551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:31.370551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:31.370551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:31.370551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:31.370551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:31.370551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:31.370551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:31.370551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:31.370551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:31.370551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:31.370551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:31.371755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:31.371755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:31.371755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:31.371755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:31.371755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:31.371755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:31.371755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:31.371755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:31.371755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:31.371755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:31.371755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:31.371755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:31.371755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:31.371755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:31.371755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:31.371755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:31.371755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:31.372929 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:31.372929 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:31.372929 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:31.374631 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 275/557] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:31.376138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767fe7d8-cfd8-40cb-84df-d6161d859876 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:31.427896 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a014d-495f-4501-97f1-541d7c10217f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:31.464270 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d6851-3432-48cf-b307-a5dee284ec51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:31.465127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d6851-3432-48cf-b307-a5dee284ec51 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:31.465514 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d6851-3432-48cf-b307-a5dee284ec51 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:31.465890 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d6851-3432-48cf-b307-a5dee284ec51 None admin] PATH_INFO: `/v3/projects/0ee65b9a8229495bbe97318f060d6342/users/eeeab506ab5c4d6386fc899086f0abf5/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:31.467904 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d6851-3432-48cf-b307-a5dee284ec51 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ee65b9a8229495bbe97318f060d6342, user_id=eeeab506ab5c4d6386fc899086f0abf5, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:31.469450 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d6851-3432-48cf-b307-a5dee284ec51 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:31.489272 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 283/558] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:52:31 2022] PUT /identity/v3/projects/0ee65b9a8229495bbe97318f060d6342/users/eeeab506ab5c4d6386fc899086f0abf5/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:52:31.490450 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d6851-3432-48cf-b307-a5dee284ec51 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:31.493322 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:31.529669 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767fe7d8-cfd8-40cb-84df-d6161d859876 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:31.578248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:31.605564 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68006079-2635-4d65-9c1a-5478f46eea38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:31.606201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68006079-2635-4d65-9c1a-5478f46eea38 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:31.606478 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68006079-2635-4d65-9c1a-5478f46eea38 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:31.606750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68006079-2635-4d65-9c1a-5478f46eea38 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:31.607643 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68006079-2635-4d65-9c1a-5478f46eea38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:31.607752 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68006079-2635-4d65-9c1a-5478f46eea38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:31.609147 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68006079-2635-4d65-9c1a-5478f46eea38 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:31.618640 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d6851-3432-48cf-b307-a5dee284ec51 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:31.633166 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 276/559] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:31.634168 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68006079-2635-4d65-9c1a-5478f46eea38 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:31.660318 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f729942-4501-476d-891a-7545638a3f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:31.661520 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f729942-4501-476d-891a-7545638a3f7c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:31.662005 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f729942-4501-476d-891a-7545638a3f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:31.662476 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f729942-4501-476d-891a-7545638a3f7c None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:31.663389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f729942-4501-476d-891a-7545638a3f7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:31.663898 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f729942-4501-476d-891a-7545638a3f7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:31.665621 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f729942-4501-476d-891a-7545638a3f7c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:31.684313 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 284/560] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:31.685589 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f729942-4501-476d-891a-7545638a3f7c None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:31.775529 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68006079-2635-4d65-9c1a-5478f46eea38 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:31.794590 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f729942-4501-476d-891a-7545638a3f7c None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:31.827722 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4da020-e5f3-49d5-a5f4-87c0b7265366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:31.829882 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4da020-e5f3-49d5-a5f4-87c0b7265366 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:31.831591 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4da020-e5f3-49d5-a5f4-87c0b7265366 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:31.832075 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4da020-e5f3-49d5-a5f4-87c0b7265366 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:31.848239 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4da020-e5f3-49d5-a5f4-87c0b7265366 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:31.848853 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:31.852542 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4da020-e5f3-49d5-a5f4-87c0b7265366 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:31.857752 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912015a9-05ba-41c1-86e9-eb2b0d6d70d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:31.858903 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912015a9-05ba-41c1-86e9-eb2b0d6d70d7 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:31.860266 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912015a9-05ba-41c1-86e9-eb2b0d6d70d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:31.864658 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912015a9-05ba-41c1-86e9-eb2b0d6d70d7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:31.879527 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-6c4da020-e5f3-49d5-a5f4-87c0b7265366 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 20 05:52:31.879527 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:31.879527 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:31.879527 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:31.879527 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:31.879527 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:31.879527 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:31.879527 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:31.879527 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:31.879527 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:31.879527 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:31.879527 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:31.879527 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:52:31.879527 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:52:31.879527 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:52:31.879527 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:52:31.880552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:31.880552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:31.880552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:52:31.880552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:31.880552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:52:31.880552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:31.880552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:31.880552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:31.880552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:31.880552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:31.880552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:31.880552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:31.880552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:31.880552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:31.880552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:31.880552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:31.880552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:31.880552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:31.881504 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:31.881504 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:31.881504 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:52:31.881504 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:52:31.881504 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:52:31.881504 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:52:31.881504 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:52:31.881504 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:52:31.881504 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 20 05:52:31.881504 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:31.882664 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912015a9-05ba-41c1-86e9-eb2b0d6d70d7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:31.884554 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912015a9-05ba-41c1-86e9-eb2b0d6d70d7 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:31.887094 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 277/561] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Oct 20 05:52:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:31.896418 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4da020-e5f3-49d5-a5f4-87c0b7265366 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:31.902622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-912015a9-05ba-41c1-86e9-eb2b0d6d70d7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 20 05:52:31.902622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:31.902622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:31.902622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:31.902622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:31.902622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:31.902622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:31.902622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:31.902622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:31.902622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:31.902622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:31.902622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:31.902622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:52:31.902622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:52:31.902622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:52:31.902622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:52:31.903660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:31.903660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:31.903660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:52:31.903660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:31.903660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:52:31.903660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:31.903660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:31.903660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:31.903660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:31.903660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:31.903660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:31.903660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:31.903660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:31.903660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:31.903660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:31.903660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:31.903660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:31.903660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:31.904618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:31.904618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:31.904618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:52:31.904618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:52:31.904618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:52:31.904618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:52:31.904618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:52:31.904618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:52:31.904618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 20 05:52:31.904618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:31.916097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 285/562] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Oct 20 05:52:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:31.916813 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:31.917094 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912015a9-05ba-41c1-86e9-eb2b0d6d70d7 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:32.048546 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912015a9-05ba-41c1-86e9-eb2b0d6d70d7 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:32.081354 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4da020-e5f3-49d5-a5f4-87c0b7265366 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:32.094664 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17b824-9f56-4d3f-a2c9-0f1b80729da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:32.095287 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17b824-9f56-4d3f-a2c9-0f1b80729da6 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:32.095583 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17b824-9f56-4d3f-a2c9-0f1b80729da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:32.096067 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17b824-9f56-4d3f-a2c9-0f1b80729da6 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:32.097184 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17b824-9f56-4d3f-a2c9-0f1b80729da6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:32.097496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17b824-9f56-4d3f-a2c9-0f1b80729da6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:32.099216 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17b824-9f56-4d3f-a2c9-0f1b80729da6 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:32.122225 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead755e7-1ab9-4d42-a598-a71ea9557ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:32.123270 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead755e7-1ab9-4d42-a598-a71ea9557ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:32.123742 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead755e7-1ab9-4d42-a598-a71ea9557ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:32.124215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead755e7-1ab9-4d42-a598-a71ea9557ea0 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:32.124326 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 286/563] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Oct 20 05:52:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:32.125364 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead755e7-1ab9-4d42-a598-a71ea9557ea0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:32.125637 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d267a-4f5d-4a9d-8a1b-6d7a9b947539 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:32.125922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d267a-4f5d-4a9d-8a1b-6d7a9b947539 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:32.126195 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d267a-4f5d-4a9d-8a1b-6d7a9b947539 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:32.126501 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead755e7-1ab9-4d42-a598-a71ea9557ea0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:32.128646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead755e7-1ab9-4d42-a598-a71ea9557ea0 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:32.132228 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 287/564] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:32.134194 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d267a-4f5d-4a9d-8a1b-6d7a9b947539 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:32.150941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 278/565] 10.222.0.20 () {56 vars in 1149 bytes} [Thu Oct 20 05:52:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:32.152571 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aad773-b81f-4b68-b16e-ecddc7a557c4 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:32.153086 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aad773-b81f-4b68-b16e-ecddc7a557c4 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:32.153571 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aad773-b81f-4b68-b16e-ecddc7a557c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:32.177029 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-09aad773-b81f-4b68-b16e-ecddc7a557c4 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:32.178918 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-09aad773-b81f-4b68-b16e-ecddc7a557c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:32.201361 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:32.246989 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:32.265137 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 279/566] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:52:32.266427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aad773-b81f-4b68-b16e-ecddc7a557c4 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:32.271916 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d267a-4f5d-4a9d-8a1b-6d7a9b947539 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:32.321713 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f65514-e859-4ada-a496-ddeec94eda77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:32.322335 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f65514-e859-4ada-a496-ddeec94eda77 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:32.322615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f65514-e859-4ada-a496-ddeec94eda77 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:32.322885 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f65514-e859-4ada-a496-ddeec94eda77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:32.399216 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f65514-e859-4ada-a496-ddeec94eda77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:32.401024 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f65514-e859-4ada-a496-ddeec94eda77 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:32.461617 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:32.476605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aad773-b81f-4b68-b16e-ecddc7a557c4 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:32.505603 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 288/567] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:52:32 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 05:52:32.517780 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f65514-e859-4ada-a496-ddeec94eda77 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:32.558925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a71da9-30cd-41bf-992a-2b72b0c8cfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:32.559486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a71da9-30cd-41bf-992a-2b72b0c8cfe0 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:32.559764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a71da9-30cd-41bf-992a-2b72b0c8cfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:32.560035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a71da9-30cd-41bf-992a-2b72b0c8cfe0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:32.580072 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a71da9-30cd-41bf-992a-2b72b0c8cfe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:32.581790 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a71da9-30cd-41bf-992a-2b72b0c8cfe0 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:32.612945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-a7a71da9-30cd-41bf-992a-2b72b0c8cfe0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:32.612945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:32.612945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:32.612945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:32.612945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:32.612945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:32.612945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:32.612945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:32.612945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:32.612945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:32.612945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:32.612945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:32.612945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:32.612945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:32.612945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:32.612945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:32.613922 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:32.613922 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:32.613922 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:32.613922 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:32.613922 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:32.613922 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:32.613922 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:32.613922 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:32.613922 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:32.613922 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:32.613922 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:32.613922 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:32.613922 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:32.613922 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:32.613922 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:32.613922 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:32.613922 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:32.614809 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:32.614809 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:32.614809 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:32.614809 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 280/568] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:32.620312 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a71da9-30cd-41bf-992a-2b72b0c8cfe0 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:32.685086 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:32.710525 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f65514-e859-4ada-a496-ddeec94eda77 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:32.746095 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79241cc-ba28-4833-9063-668b047e79db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:32.746779 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79241cc-ba28-4833-9063-668b047e79db None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:32.747065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79241cc-ba28-4833-9063-668b047e79db None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:32.747339 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79241cc-ba28-4833-9063-668b047e79db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:32.755229 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a71da9-30cd-41bf-992a-2b72b0c8cfe0 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:32.757841 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79241cc-ba28-4833-9063-668b047e79db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:32.759361 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79241cc-ba28-4833-9063-668b047e79db None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:32.775405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-c79241cc-ba28-4833-9063-668b047e79db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:32.775405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:32.775405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:32.775405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:32.775405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:32.775405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:32.775405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:32.775405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:32.775405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:32.775405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:32.775405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:32.775405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:32.775405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:32.775405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:32.775405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:32.775405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:32.776432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:32.776432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:32.776432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:32.776432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:32.776432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:32.776432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:32.776432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:32.776432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:32.776432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:32.776432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:32.776432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:32.776432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:32.776432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:32.776432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:32.776432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:32.776432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:32.776432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:32.777365 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:32.777365 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:32.777365 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:32.781183 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 289/569] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:32.782096 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79241cc-ba28-4833-9063-668b047e79db None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:32.785749 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:32.798390 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab662c-8ea6-4b26-b3c8-49f8778570ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:32.799156 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab662c-8ea6-4b26-b3c8-49f8778570ac None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:32.799461 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab662c-8ea6-4b26-b3c8-49f8778570ac None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:32.799751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab662c-8ea6-4b26-b3c8-49f8778570ac None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:32.800869 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab662c-8ea6-4b26-b3c8-49f8778570ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:32.801352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab662c-8ea6-4b26-b3c8-49f8778570ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:32.803167 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab662c-8ea6-4b26-b3c8-49f8778570ac None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:32.820302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 281/570] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:32.821145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab662c-8ea6-4b26-b3c8-49f8778570ac None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:32.910090 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79241cc-ba28-4833-9063-668b047e79db None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:32.946942 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b9b94-d222-4158-9129-97bca35eaaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:32.947580 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b9b94-d222-4158-9129-97bca35eaaf8 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:32.947855 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b9b94-d222-4158-9129-97bca35eaaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:32.948127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b9b94-d222-4158-9129-97bca35eaaf8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:32.948728 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b9b94-d222-4158-9129-97bca35eaaf8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:32.950083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b9b94-d222-4158-9129-97bca35eaaf8 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:32.962076 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-dd4b9b94-d222-4158-9129-97bca35eaaf8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 05:52:32.962076 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:32.962076 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:52:32.962076 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:52:32.962076 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:52:32.962076 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:52:32.962076 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:52:32.962076 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:32.962076 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 05:52:32.962076 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:32.962076 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:52:32.962076 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:32.962076 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:32.962076 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:32.962076 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:32.962076 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:32.962076 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:32.963140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:32.963140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:32.963140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:32.963140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:32.963140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:32.963140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:32.963140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:52:32.963140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:52:32.963140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:52:32.963140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:52:32.963140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:32.963140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:32.963140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:32.963140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:32.963140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:32.963140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:32.964602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:32.964602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:32.964602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:32.964602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:32.964602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:32.964602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:32.964602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:32.964602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:32.964602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:32.964602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:32.964602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:52:32.964602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:32.964602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 05:52:32.964602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:32.964602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 290/571] 10.222.0.20 () {56 vars in 1129 bytes} [Thu Oct 20 05:52:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:32.965456 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b9b94-d222-4158-9129-97bca35eaaf8 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:32.972125 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab662c-8ea6-4b26-b3c8-49f8778570ac None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:32.985750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:33.023347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9029d4-0f0c-4760-8cec-dc1ecd19e18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:33.024633 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9029d4-0f0c-4760-8cec-dc1ecd19e18b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:33.025489 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9029d4-0f0c-4760-8cec-dc1ecd19e18b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:33.026214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9029d4-0f0c-4760-8cec-dc1ecd19e18b None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:33.027309 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9029d4-0f0c-4760-8cec-dc1ecd19e18b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:33.027797 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9029d4-0f0c-4760-8cec-dc1ecd19e18b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:33.030065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9029d4-0f0c-4760-8cec-dc1ecd19e18b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:33.034822 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:33.047945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 282/572] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:33.053640 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9029d4-0f0c-4760-8cec-dc1ecd19e18b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:33.084954 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b9b94-d222-4158-9129-97bca35eaaf8 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:33.132912 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6b13b-86e6-44cc-a08c-37b560e2f243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:33.133881 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6b13b-86e6-44cc-a08c-37b560e2f243 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:33.134642 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6b13b-86e6-44cc-a08c-37b560e2f243 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:33.135411 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6b13b-86e6-44cc-a08c-37b560e2f243 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:33.145709 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6b13b-86e6-44cc-a08c-37b560e2f243 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:33.147387 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6b13b-86e6-44cc-a08c-37b560e2f243 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:33.163976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-7dc6b13b-86e6-44cc-a08c-37b560e2f243 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:33.163976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:33.163976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:33.163976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:33.163976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:33.163976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:33.163976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:33.163976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:33.163976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:33.163976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:33.163976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:33.163976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:33.163976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:52:33.163976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:52:33.163976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:52:33.164960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:52:33.164960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:33.164960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:33.164960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:33.164960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:33.164960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:33.164960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:33.164960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:33.164960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:33.164960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:33.164960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:33.164960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:33.164960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:33.164960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:33.164960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:33.164960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:33.164960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:33.165846 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:52:33.165846 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:52:33.165846 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:52:33.165846 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:52:33.165846 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:52:33.165846 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:52:33.165846 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:33.165846 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:33.168845 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 291/573] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:33.170148 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6b13b-86e6-44cc-a08c-37b560e2f243 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:33.177979 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9029d4-0f0c-4760-8cec-dc1ecd19e18b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:33.220530 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27384364-557e-453e-bf40-697b3e0e51a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:33.222098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27384364-557e-453e-bf40-697b3e0e51a8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:33.222525 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27384364-557e-453e-bf40-697b3e0e51a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:33.222942 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27384364-557e-453e-bf40-697b3e0e51a8 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:33.224275 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27384364-557e-453e-bf40-697b3e0e51a8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:33.224760 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27384364-557e-453e-bf40-697b3e0e51a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:33.226731 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27384364-557e-453e-bf40-697b3e0e51a8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:33.233427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 283/574] 10.222.0.20 () {56 vars in 1124 bytes} [Thu Oct 20 05:52:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:33.234790 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27384364-557e-453e-bf40-697b3e0e51a8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:33.278477 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:33.291433 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6b13b-86e6-44cc-a08c-37b560e2f243 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:33.322103 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:33.342978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58b246-cab3-4f3e-a59c-193e0afa0972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:33.343849 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58b246-cab3-4f3e-a59c-193e0afa0972 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:33.344772 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58b246-cab3-4f3e-a59c-193e0afa0972 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:33.345221 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58b246-cab3-4f3e-a59c-193e0afa0972 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:33.355799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef58b246-cab3-4f3e-a59c-193e0afa0972 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:33.358292 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef58b246-cab3-4f3e-a59c-193e0afa0972 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:33.371819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-ef58b246-cab3-4f3e-a59c-193e0afa0972 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:33.371819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:33.371819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:33.371819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:33.371819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:33.371819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:33.371819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:33.371819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:33.371819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:33.371819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:33.371819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:33.371819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:33.371819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:52:33.371819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:52:33.371819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:52:33.373031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:52:33.373031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:33.373031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:33.373031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:33.373031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:33.373031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:33.373031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:33.373031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:33.373031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:33.373031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:33.373031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:33.373031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:33.373031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:33.373031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:33.373031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:33.373031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:33.373031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:33.374319 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:52:33.374319 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:52:33.374319 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:52:33.374319 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:52:33.374319 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:52:33.374319 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:52:33.374319 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:33.374319 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:33.375960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 292/575] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:33.378004 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27384364-557e-453e-bf40-697b3e0e51a8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:33.378149 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58b246-cab3-4f3e-a59c-193e0afa0972 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:33.430869 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcc27f-df47-46cd-b3f4-f521d3dcb878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:33.431553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcc27f-df47-46cd-b3f4-f521d3dcb878 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:33.431926 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcc27f-df47-46cd-b3f4-f521d3dcb878 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:33.432311 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcc27f-df47-46cd-b3f4-f521d3dcb878 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:33.433127 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bcc27f-df47-46cd-b3f4-f521d3dcb878 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:33.433514 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bcc27f-df47-46cd-b3f4-f521d3dcb878 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:33.435020 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bcc27f-df47-46cd-b3f4-f521d3dcb878 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:33.448408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 284/576] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Oct 20 05:52:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:33.449508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcc27f-df47-46cd-b3f4-f521d3dcb878 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:33.508414 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58b246-cab3-4f3e-a59c-193e0afa0972 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:33.545314 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d740b2-c102-412a-b3c9-f70cf8ec7420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:33.545929 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d740b2-c102-412a-b3c9-f70cf8ec7420 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:33.546211 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d740b2-c102-412a-b3c9-f70cf8ec7420 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:33.546485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d740b2-c102-412a-b3c9-f70cf8ec7420 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:33.547109 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d740b2-c102-412a-b3c9-f70cf8ec7420 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:33.547409 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d740b2-c102-412a-b3c9-f70cf8ec7420 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:33.547975 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:33.548837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d740b2-c102-412a-b3c9-f70cf8ec7420 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:33.556378 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 293/577] 10.222.0.20 () {56 vars in 1124 bytes} [Thu Oct 20 05:52:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:33.558195 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d740b2-c102-412a-b3c9-f70cf8ec7420 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:33.564466 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcc27f-df47-46cd-b3f4-f521d3dcb878 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:33.599148 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70414057-dc29-45d1-8a3d-832fdb24b3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:33.599681 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70414057-dc29-45d1-8a3d-832fdb24b3e9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:33.599960 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70414057-dc29-45d1-8a3d-832fdb24b3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:33.600233 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70414057-dc29-45d1-8a3d-832fdb24b3e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:33.600947 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70414057-dc29-45d1-8a3d-832fdb24b3e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:33.601254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70414057-dc29-45d1-8a3d-832fdb24b3e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:33.602678 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70414057-dc29-45d1-8a3d-832fdb24b3e9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:33.613898 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:33.615278 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 285/578] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Oct 20 05:52:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:33.616254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c4fc8-1a02-4751-abb4-54e0b5d45c2a None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:33.616545 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c4fc8-1a02-4751-abb4-54e0b5d45c2a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:33.616856 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c4fc8-1a02-4751-abb4-54e0b5d45c2a None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:33.618189 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 286/579] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:33.622736 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c4fc8-1a02-4751-abb4-54e0b5d45c2a None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:33.674693 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d740b2-c102-412a-b3c9-f70cf8ec7420 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:33.712272 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d8e5b-85cc-4a73-ae9a-9f4be8c3f25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:33.712969 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d8e5b-85cc-4a73-ae9a-9f4be8c3f25b None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:33.713251 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d8e5b-85cc-4a73-ae9a-9f4be8c3f25b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:33.713524 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d8e5b-85cc-4a73-ae9a-9f4be8c3f25b None admin] PATH_INFO: `/v3/projects/0ee65b9a8229495bbe97318f060d6342/users/7a59f547a20e48ee88e3e620ba1e87f1/roles/afd859d2deed4d51aa3f6326920441c8` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:33.728996 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c4fc8-1a02-4751-abb4-54e0b5d45c2a None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:33.731117 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477d8e5b-85cc-4a73-ae9a-9f4be8c3f25b None admin] RBAC: Authorizing `identity:create_grant(project_id=0ee65b9a8229495bbe97318f060d6342, user_id=7a59f547a20e48ee88e3e620ba1e87f1, role_id=afd859d2deed4d51aa3f6326920441c8)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:33.732571 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477d8e5b-85cc-4a73-ae9a-9f4be8c3f25b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:33.760740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 294/580] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:52:33 2022] PUT /identity/v3/projects/0ee65b9a8229495bbe97318f060d6342/users/7a59f547a20e48ee88e3e620ba1e87f1/roles/afd859d2deed4d51aa3f6326920441c8 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:52:33.760740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d8e5b-85cc-4a73-ae9a-9f4be8c3f25b None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:33.778074 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3da189-e4c2-42fa-b84e-b036f22d9846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:33.778797 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3da189-e4c2-42fa-b84e-b036f22d9846 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:33.779164 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3da189-e4c2-42fa-b84e-b036f22d9846 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:33.779497 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3da189-e4c2-42fa-b84e-b036f22d9846 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:33.780286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3da189-e4c2-42fa-b84e-b036f22d9846 None admin] RBAC: Adding query filter params (interface=public, service_id=748d0842660940dab0c4ceda6105ff6e, region_id=RegionOne) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:33.780662 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3da189-e4c2-42fa-b84e-b036f22d9846 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:33.782196 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3da189-e4c2-42fa-b84e-b036f22d9846 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:33.795666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 287/581] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Oct 20 05:52:33 2022] GET /identity/v3/endpoints?service_id=748d0842660940dab0c4ceda6105ff6e&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:33.797106 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8214842-8f59-428f-9c7e-cbf54673683f None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:33.797445 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8214842-8f59-428f-9c7e-cbf54673683f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:33.797788 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8214842-8f59-428f-9c7e-cbf54673683f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:33.817125 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-c8214842-8f59-428f-9c7e-cbf54673683f None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:33.818436 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c8214842-8f59-428f-9c7e-cbf54673683f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:33.830947 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:33.894776 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:33.896478 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 288/582] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:52:33.899294 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d8e5b-85cc-4a73-ae9a-9f4be8c3f25b None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:33.904244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8214842-8f59-428f-9c7e-cbf54673683f None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:33.949517 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ed64f-24b3-41c2-9cf5-e74a94d46e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:33.950148 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ed64f-24b3-41c2-9cf5-e74a94d46e70 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:33.950425 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ed64f-24b3-41c2-9cf5-e74a94d46e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:33.950695 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ed64f-24b3-41c2-9cf5-e74a94d46e70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:33.951424 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ed64f-24b3-41c2-9cf5-e74a94d46e70 None admin] RBAC: Adding query filter params (role.id=d8debbb193834fac9cc25f81726f3a67, scope.project.id=0ee65b9a8229495bbe97318f060d6342, user.id=ae2c1c886cd94d9f90082f66e7e8c9f4) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:33.951734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ed64f-24b3-41c2-9cf5-e74a94d46e70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:33.953206 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ed64f-24b3-41c2-9cf5-e74a94d46e70 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:33.961132 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 295/583] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:52:33 2022] GET /identity/v3/role_assignments?user.id=ae2c1c886cd94d9f90082f66e7e8c9f4&scope.project.id=0ee65b9a8229495bbe97318f060d6342&role.id=d8debbb193834fac9cc25f81726f3a67 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:33.962282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce05b1f-2336-434d-9eee-74fae23fdbff None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:33.962638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce05b1f-2336-434d-9eee-74fae23fdbff None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:33.962922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce05b1f-2336-434d-9eee-74fae23fdbff None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:33.964421 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 296/584] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:33.970583 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab410def-3cf6-467a-b757-8a786adf2a38 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:33.971474 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab410def-3cf6-467a-b757-8a786adf2a38 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:33.971901 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab410def-3cf6-467a-b757-8a786adf2a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:33.992678 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-ab410def-3cf6-467a-b757-8a786adf2a38 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:33.994763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ab410def-3cf6-467a-b757-8a786adf2a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:34.052648 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8214842-8f59-428f-9c7e-cbf54673683f None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:34.085012 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 297/585] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 05:52:34.101790 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271cbe07-805f-4f1a-a90a-f55416a39601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:34.102762 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cbe07-805f-4f1a-a90a-f55416a39601 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:34.103331 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cbe07-805f-4f1a-a90a-f55416a39601 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:34.103802 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cbe07-805f-4f1a-a90a-f55416a39601 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:34.104664 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271cbe07-805f-4f1a-a90a-f55416a39601 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:34.106334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271cbe07-805f-4f1a-a90a-f55416a39601 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:34.113321 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 289/586] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Oct 20 05:52:33 2022] GET /identity/v3/services => generated 772 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:34.207772 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab410def-3cf6-467a-b757-8a786adf2a38 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:34.332423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:34.341062 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab410def-3cf6-467a-b757-8a786adf2a38 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:34.379881 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ae456-8179-4454-ae3c-3838674cffb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:34.380955 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ae456-8179-4454-ae3c-3838674cffb6 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:34.381464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ae456-8179-4454-ae3c-3838674cffb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:34.381943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ae456-8179-4454-ae3c-3838674cffb6 None admin] PATH_INFO: `/v3/services/network` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:34.382735 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ae456-8179-4454-ae3c-3838674cffb6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:34.384428 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ae456-8179-4454-ae3c-3838674cffb6 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:34.395050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271cbe07-805f-4f1a-a90a-f55416a39601 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:34.396240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-403ae456-8179-4454-ae3c-3838674cffb6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 05:52:34.396240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:34.396240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:52:34.396240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:52:34.396240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:52:34.396240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:52:34.396240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:52:34.396240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:34.396240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 05:52:34.396240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:34.396240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:52:34.396240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:34.396240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:34.396240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:34.396240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:34.396240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:34.396240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:34.397480 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:34.397480 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:34.397480 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:34.397480 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:34.397480 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:34.397480 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:34.397480 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:52:34.397480 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:52:34.397480 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:52:34.397480 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:52:34.397480 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:34.397480 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:34.397480 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:34.397480 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:34.397480 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:34.397480 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:34.398615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:34.398615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:34.398615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:34.398615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:34.398615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:34.398615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:34.398615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:34.398615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:34.398615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:34.398615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:34.398615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:52:34.398615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:34.398615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 05:52:34.398615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:34.401186 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 298/587] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:34 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:34.407056 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ae456-8179-4454-ae3c-3838674cffb6 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:34.416248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:34.546699 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ae456-8179-4454-ae3c-3838674cffb6 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:34.580632 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271cbe07-805f-4f1a-a90a-f55416a39601 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:34.611408 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd18fc-228d-4d28-a846-53f5e7595825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:34.612359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd18fc-228d-4d28-a846-53f5e7595825 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:34.612640 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd18fc-228d-4d28-a846-53f5e7595825 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:34.612943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd18fc-228d-4d28-a846-53f5e7595825 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:34.613587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd18fc-228d-4d28-a846-53f5e7595825 None admin] RBAC: Adding query filter params (name=network) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:34.613894 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd18fc-228d-4d28-a846-53f5e7595825 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:34.615274 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd18fc-228d-4d28-a846-53f5e7595825 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:34.625205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 299/588] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Oct 20 05:52:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:34.632470 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd18fc-228d-4d28-a846-53f5e7595825 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:34.649477 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c94bdfc-3fcb-4b15-b111-d96fbc302ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:34.650119 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94bdfc-3fcb-4b15-b111-d96fbc302ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:34.650396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94bdfc-3fcb-4b15-b111-d96fbc302ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:34.650667 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94bdfc-3fcb-4b15-b111-d96fbc302ba8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:34.669060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c94bdfc-3fcb-4b15-b111-d96fbc302ba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:52:34.680884 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:34.681955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-4c94bdfc-3fcb-4b15-b111-d96fbc302ba8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:52:34.681955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:34.681955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:34.681955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:34.681955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:34.681955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:34.681955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:34.681955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:34.681955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:34.681955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:34.681955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:34.681955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:34.681955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:52:34.681955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:52:34.681955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:52:34.681955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:52:34.683009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:52:34.683009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:52:34.683009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:52:34.683009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:52:34.683009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:34.683009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:34.683009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:34.683009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:34.683009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:34.683009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:34.683009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:34.683009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:34.683009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:34.683009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:34.683009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:34.683009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:34.683864 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:34.683864 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:34.683864 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:34.683864 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:34.683864 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:52:34.683864 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:52:34.683864 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:52:34.683864 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:52:34.683864 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:52:34.683864 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:52:34.683864 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:52:34.683864 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:34.685628 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 290/589] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Oct 20 05:52:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:34.691956 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c94bdfc-3fcb-4b15-b111-d96fbc302ba8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:34.816554 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd18fc-228d-4d28-a846-53f5e7595825 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:34.837446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:34.857694 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e82c1f-20ec-48fc-962f-d3ca2715c6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:34.858415 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e82c1f-20ec-48fc-962f-d3ca2715c6ed None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:34.858706 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e82c1f-20ec-48fc-962f-d3ca2715c6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:34.858980 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e82c1f-20ec-48fc-962f-d3ca2715c6ed None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:34.859609 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e82c1f-20ec-48fc-962f-d3ca2715c6ed None admin] RBAC: Adding query filter params (type=network) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:34.860013 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e82c1f-20ec-48fc-962f-d3ca2715c6ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:34.861601 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e82c1f-20ec-48fc-962f-d3ca2715c6ed None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:34.867455 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 300/590] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Oct 20 05:52:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:34.873659 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e82c1f-20ec-48fc-962f-d3ca2715c6ed None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:34.902188 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c94bdfc-3fcb-4b15-b111-d96fbc302ba8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:34.951906 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887877e-4a4a-4d5e-a1f3-878ebb76daeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:34.952900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887877e-4a4a-4d5e-a1f3-878ebb76daeb None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:34.953411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887877e-4a4a-4d5e-a1f3-878ebb76daeb None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:34.953881 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887877e-4a4a-4d5e-a1f3-878ebb76daeb None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:34.954756 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887877e-4a4a-4d5e-a1f3-878ebb76daeb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:34.955781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887877e-4a4a-4d5e-a1f3-878ebb76daeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:34.957571 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887877e-4a4a-4d5e-a1f3-878ebb76daeb None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:34.970487 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 291/591] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:52:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:34.973209 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887877e-4a4a-4d5e-a1f3-878ebb76daeb None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:35.024586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:35.052921 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e82c1f-20ec-48fc-962f-d3ca2715c6ed None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:35.101948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ca4a1-bba7-4aed-9ee5-a4f6116ed84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:35.102484 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ca4a1-bba7-4aed-9ee5-a4f6116ed84c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:35.102763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ca4a1-bba7-4aed-9ee5-a4f6116ed84c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:35.103038 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ca4a1-bba7-4aed-9ee5-a4f6116ed84c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:35.103736 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ca4a1-bba7-4aed-9ee5-a4f6116ed84c None admin] RBAC: Adding query filter params (interface=public, service_id=0084859a924049e8b5542641e6baa54e, region_id=RegionOne) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:35.104047 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ca4a1-bba7-4aed-9ee5-a4f6116ed84c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:35.105454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ca4a1-bba7-4aed-9ee5-a4f6116ed84c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:35.119956 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 301/592] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Oct 20 05:52:34 2022] GET /identity/v3/endpoints?service_id=0084859a924049e8b5542641e6baa54e&interface=public®ion_id=RegionOne => generated 194 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:35.125210 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ca4a1-bba7-4aed-9ee5-a4f6116ed84c None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:35.147815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887877e-4a4a-4d5e-a1f3-878ebb76daeb None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:35.186239 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e146175-19d6-405a-95db-888f9c22ae11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:35.186880 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e146175-19d6-405a-95db-888f9c22ae11 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:35.187159 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e146175-19d6-405a-95db-888f9c22ae11 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:35.187430 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e146175-19d6-405a-95db-888f9c22ae11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:35.197111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:35.199787 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e146175-19d6-405a-95db-888f9c22ae11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:35.201339 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e146175-19d6-405a-95db-888f9c22ae11 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:35.217828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-7e146175-19d6-405a-95db-888f9c22ae11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:35.217828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:35.217828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:35.217828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:35.217828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:35.217828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:35.217828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:35.217828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:35.217828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:35.217828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:35.217828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:35.217828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:35.217828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:35.217828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:35.217828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:35.217828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:35.218812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:35.218812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:35.218812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:35.218812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:35.218812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:35.218812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:35.218812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:35.218812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:35.218812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:35.218812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:35.218812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:35.218812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:35.218812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:35.218812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:35.218812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:35.218812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:35.218812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:35.219990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:35.219990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:35.219990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:35.219990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 292/593] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:35.226457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e146175-19d6-405a-95db-888f9c22ae11 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:35.403500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ca4a1-bba7-4aed-9ee5-a4f6116ed84c None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:35.405566 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e146175-19d6-405a-95db-888f9c22ae11 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:35.443919 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ac4fd-44c5-48e2-97c5-6f807d5026b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:35.444536 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ac4fd-44c5-48e2-97c5-6f807d5026b9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:35.444839 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ac4fd-44c5-48e2-97c5-6f807d5026b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:35.445246 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ac4fd-44c5-48e2-97c5-6f807d5026b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:35.445940 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ac4fd-44c5-48e2-97c5-6f807d5026b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:35.446252 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ac4fd-44c5-48e2-97c5-6f807d5026b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:35.447619 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ac4fd-44c5-48e2-97c5-6f807d5026b9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:35.462115 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 293/594] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:35.469641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ac4fd-44c5-48e2-97c5-6f807d5026b9 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:35.508211 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f82ce-febe-4a3d-8c65-f9117f27c95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:35.509219 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f82ce-febe-4a3d-8c65-f9117f27c95f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:35.510173 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f82ce-febe-4a3d-8c65-f9117f27c95f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:35.511276 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f82ce-febe-4a3d-8c65-f9117f27c95f None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:35.512090 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f82ce-febe-4a3d-8c65-f9117f27c95f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:35.516008 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f82ce-febe-4a3d-8c65-f9117f27c95f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:35.527820 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 302/595] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Oct 20 05:52:35 2022] GET /identity/v3/services => generated 772 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:35.532357 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:35.616543 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ac4fd-44c5-48e2-97c5-6f807d5026b9 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:35.654430 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecb5e0-8361-448e-be85-59dd80d0bfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:35.655438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecb5e0-8361-448e-be85-59dd80d0bfa6 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:35.655926 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecb5e0-8361-448e-be85-59dd80d0bfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:35.656400 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecb5e0-8361-448e-be85-59dd80d0bfa6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:35.664878 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cecb5e0-8361-448e-be85-59dd80d0bfa6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:35.666710 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cecb5e0-8361-448e-be85-59dd80d0bfa6 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:35.680687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-5cecb5e0-8361-448e-be85-59dd80d0bfa6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 20 05:52:35.680687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:35.680687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:35.680687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:35.680687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:35.680687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:35.680687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:35.680687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:35.680687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:35.680687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:35.680687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:35.680687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:35.680687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:52:35.680687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:52:35.680687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:52:35.680687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:52:35.681763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:35.681763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:35.681763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:52:35.681763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:35.681763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:52:35.681763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:35.681763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:35.681763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:35.681763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:35.681763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:35.681763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:35.681763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:35.681763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:35.681763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:35.681763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:35.681763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:35.681763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:35.681763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:35.682713 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:35.682713 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:35.682713 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:52:35.682713 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:52:35.682713 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:52:35.682713 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:52:35.682713 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:52:35.682713 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:52:35.682713 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 20 05:52:35.682713 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:35.688794 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 294/596] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Oct 20 05:52:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:35.691542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f82ce-febe-4a3d-8c65-f9117f27c95f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:35.730730 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:35.837569 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f82ce-febe-4a3d-8c65-f9117f27c95f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:35.876651 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32c5cc-4ae2-43d0-bb69-31cc0ae4c5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:35.877528 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32c5cc-4ae2-43d0-bb69-31cc0ae4c5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:35.877804 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32c5cc-4ae2-43d0-bb69-31cc0ae4c5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:35.878081 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32c5cc-4ae2-43d0-bb69-31cc0ae4c5f7 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:35.878872 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a32c5cc-4ae2-43d0-bb69-31cc0ae4c5f7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:35.879177 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a32c5cc-4ae2-43d0-bb69-31cc0ae4c5f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:35.880667 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a32c5cc-4ae2-43d0-bb69-31cc0ae4c5f7 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:35.899206 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 303/597] 10.222.0.20 () {56 vars in 1135 bytes} [Thu Oct 20 05:52:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:35.905089 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecb5e0-8361-448e-be85-59dd80d0bfa6 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:36.012011 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecb5e0-8361-448e-be85-59dd80d0bfa6 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:36.014363 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:36.048412 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bceb19-82d2-4a23-b0d1-2d2742037ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:36.049164 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bceb19-82d2-4a23-b0d1-2d2742037ccb None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:36.049440 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bceb19-82d2-4a23-b0d1-2d2742037ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:36.049710 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bceb19-82d2-4a23-b0d1-2d2742037ccb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:36.119569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bceb19-82d2-4a23-b0d1-2d2742037ccb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:36.121109 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bceb19-82d2-4a23-b0d1-2d2742037ccb None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:36.225485 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 295/598] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:52:35 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 20 05:52:36.233924 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32c5cc-4ae2-43d0-bb69-31cc0ae4c5f7 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:36.358661 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:36.380685 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb64f9b-d61b-4d23-8c86-ed5639ab1039 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:36.381079 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb64f9b-d61b-4d23-8c86-ed5639ab1039 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:36.381522 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb64f9b-d61b-4d23-8c86-ed5639ab1039 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:36.382930 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 296/599] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:52:36.388121 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4ef64-6d7e-4dd9-824d-f5312007d5d4 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:36.388401 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4ef64-6d7e-4dd9-824d-f5312007d5d4 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:36.388675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4ef64-6d7e-4dd9-824d-f5312007d5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:36.401094 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:36.403740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32c5cc-4ae2-43d0-bb69-31cc0ae4c5f7 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:36.422818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-3fd4ef64-6d7e-4dd9-824d-f5312007d5d4 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:36.426258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3fd4ef64-6d7e-4dd9-824d-f5312007d5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:36.469999 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115fa41-c0ae-4fa8-aa01-a01202709294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:36.470668 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115fa41-c0ae-4fa8-aa01-a01202709294 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:36.470999 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115fa41-c0ae-4fa8-aa01-a01202709294 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:36.471389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115fa41-c0ae-4fa8-aa01-a01202709294 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:36.493390 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115fa41-c0ae-4fa8-aa01-a01202709294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:36.494765 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115fa41-c0ae-4fa8-aa01-a01202709294 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:36.520840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-6115fa41-c0ae-4fa8-aa01-a01202709294 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:36.520840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:36.520840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:36.520840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:36.520840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:36.520840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:36.520840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:36.520840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:36.520840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:36.520840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:36.520840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:36.520840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:36.520840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:36.520840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:36.520840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:36.520840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:36.522051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:36.522051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:36.522051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:36.522051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:36.522051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:36.522051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:36.522051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:36.522051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:36.522051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:36.522051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:36.522051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:36.522051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:36.522051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:36.522051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:36.522051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:36.522051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:36.522051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:36.523034 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:36.523034 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:36.523034 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:36.523034 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 304/600] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:36.529716 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115fa41-c0ae-4fa8-aa01-a01202709294 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:36.553295 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 297/601] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 05:52:36.685171 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115fa41-c0ae-4fa8-aa01-a01202709294 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:36.719080 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:36.734344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3f08f-4d1c-4f8e-a4d9-48f625f8ed32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:36.734968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3f08f-4d1c-4f8e-a4d9-48f625f8ed32 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:36.735247 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3f08f-4d1c-4f8e-a4d9-48f625f8ed32 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:36.735520 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3f08f-4d1c-4f8e-a4d9-48f625f8ed32 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:36.736156 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb3f08f-4d1c-4f8e-a4d9-48f625f8ed32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:36.737040 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb3f08f-4d1c-4f8e-a4d9-48f625f8ed32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:36.738421 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb3f08f-4d1c-4f8e-a4d9-48f625f8ed32 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:36.760359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 305/602] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:36.765926 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4ef64-6d7e-4dd9-824d-f5312007d5d4 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:36.771717 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfeef7-35c3-4c41-972a-46d28e1379aa None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:36.772010 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfeef7-35c3-4c41-972a-46d28e1379aa None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:36.772291 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfeef7-35c3-4c41-972a-46d28e1379aa None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:36.773892 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 306/603] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:52:36.782219 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d094814-5457-4b29-9978-b4f5820b6336 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:36.782501 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d094814-5457-4b29-9978-b4f5820b6336 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:36.783290 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d094814-5457-4b29-9978-b4f5820b6336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:36.816335 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-8d094814-5457-4b29-9978-b4f5820b6336 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:36.817622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8d094814-5457-4b29-9978-b4f5820b6336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:36.938178 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 307/604] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 05:52:36.940866 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4ef64-6d7e-4dd9-824d-f5312007d5d4 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:36.943113 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d094814-5457-4b29-9978-b4f5820b6336 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:36.981493 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a82f9e8-7988-46db-84f0-fc87948b7c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:36.982129 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82f9e8-7988-46db-84f0-fc87948b7c36 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:36.982406 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82f9e8-7988-46db-84f0-fc87948b7c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:36.982675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82f9e8-7988-46db-84f0-fc87948b7c36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:36.986956 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:36.995612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a82f9e8-7988-46db-84f0-fc87948b7c36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:36.997503 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a82f9e8-7988-46db-84f0-fc87948b7c36 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:37.014468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-9a82f9e8-7988-46db-84f0-fc87948b7c36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:37.014468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:37.014468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:37.014468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:37.014468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:37.014468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:37.014468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:37.014468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:37.014468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:37.014468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:37.014468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:37.014468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:37.014468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:52:37.014468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:52:37.014468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:52:37.015365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:52:37.015365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:37.015365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:37.015365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:37.015365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:37.015365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:37.015365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:37.015365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:37.015365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:37.015365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:37.015365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:37.015365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:37.015365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:37.015365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:37.015365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:37.015365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:37.015365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:37.016242 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:52:37.016242 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:52:37.016242 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:52:37.016242 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:52:37.016242 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:52:37.016242 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:52:37.016242 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:37.016242 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:37.016675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 298/605] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:37.018600 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa3c77-79aa-41b0-9eb6-b08300b9ac3e None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:37.018885 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa3c77-79aa-41b0-9eb6-b08300b9ac3e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:37.019162 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa3c77-79aa-41b0-9eb6-b08300b9ac3e None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:37.020678 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 299/606] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:37.021332 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa3c77-79aa-41b0-9eb6-b08300b9ac3e None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:37.060230 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d094814-5457-4b29-9978-b4f5820b6336 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:37.078786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:37.106386 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6162d-2571-4679-a84d-ff384721826b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:37.107933 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6162d-2571-4679-a84d-ff384721826b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:37.108613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6162d-2571-4679-a84d-ff384721826b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:37.109133 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6162d-2571-4679-a84d-ff384721826b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:37.123140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6162d-2571-4679-a84d-ff384721826b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:52:37.129992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-7bb6162d-2571-4679-a84d-ff384721826b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:52:37.129992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:37.129992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:37.129992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:37.129992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:37.129992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:37.129992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:37.129992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:37.129992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:37.129992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:37.129992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:37.129992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:37.129992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:52:37.129992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:52:37.129992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:52:37.129992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:52:37.130981 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:52:37.130981 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:52:37.130981 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:52:37.130981 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:52:37.130981 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:37.130981 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:37.130981 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:37.130981 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:37.130981 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:37.130981 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:37.130981 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:37.130981 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:37.130981 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:37.130981 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:37.130981 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:37.130981 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:37.131837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:37.131837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:37.131837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:37.131837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:37.131837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:52:37.131837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:52:37.131837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:52:37.131837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:52:37.131837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:52:37.131837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:52:37.131837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:52:37.131837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:37.140038 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 308/607] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Oct 20 05:52:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:37.140038 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f883dd-c4d7-4c76-8fca-2ba96079e0c2 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:37.140038 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f883dd-c4d7-4c76-8fca-2ba96079e0c2 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:37.140038 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f883dd-c4d7-4c76-8fca-2ba96079e0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:37.153775 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa3c77-79aa-41b0-9eb6-b08300b9ac3e None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:37.164502 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-80f883dd-c4d7-4c76-8fca-2ba96079e0c2 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:37.166454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-80f883dd-c4d7-4c76-8fca-2ba96079e0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:37.197750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd79735-f080-4e89-82a8-13a7fe89cab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:37.199291 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd79735-f080-4e89-82a8-13a7fe89cab9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:37.199744 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd79735-f080-4e89-82a8-13a7fe89cab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:37.200184 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd79735-f080-4e89-82a8-13a7fe89cab9 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:37.201708 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd79735-f080-4e89-82a8-13a7fe89cab9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:37.202210 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd79735-f080-4e89-82a8-13a7fe89cab9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:37.204284 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd79735-f080-4e89-82a8-13a7fe89cab9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:37.221605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 300/608] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Oct 20 05:52:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:37.222722 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd79735-f080-4e89-82a8-13a7fe89cab9 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:37.250085 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 309/609] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:52:37.250944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f883dd-c4d7-4c76-8fca-2ba96079e0c2 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:37.295786 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:37.356975 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd79735-f080-4e89-82a8-13a7fe89cab9 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:37.373432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f883dd-c4d7-4c76-8fca-2ba96079e0c2 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:37.375907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:37.394495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4e20e-c77c-4ad8-88bf-d4b21441eda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:37.395140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4e20e-c77c-4ad8-88bf-d4b21441eda4 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:37.395421 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4e20e-c77c-4ad8-88bf-d4b21441eda4 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:37.395694 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4e20e-c77c-4ad8-88bf-d4b21441eda4 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:37.396351 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4e20e-c77c-4ad8-88bf-d4b21441eda4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:37.396657 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4e20e-c77c-4ad8-88bf-d4b21441eda4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:37.398101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4e20e-c77c-4ad8-88bf-d4b21441eda4 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:37.406838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 301/610] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Oct 20 05:52:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:37.407728 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4e20e-c77c-4ad8-88bf-d4b21441eda4 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:37.421961 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2599b9d-337e-4402-a5d8-8d0549e4068c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:37.422580 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2599b9d-337e-4402-a5d8-8d0549e4068c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:37.422921 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2599b9d-337e-4402-a5d8-8d0549e4068c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:37.423212 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2599b9d-337e-4402-a5d8-8d0549e4068c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:37.424794 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2599b9d-337e-4402-a5d8-8d0549e4068c None admin] RBAC: Adding query filter params (role.id=d8debbb193834fac9cc25f81726f3a67, scope.project.id=0ee65b9a8229495bbe97318f060d6342, user.id=eeeab506ab5c4d6386fc899086f0abf5) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:37.425109 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2599b9d-337e-4402-a5d8-8d0549e4068c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:37.426586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2599b9d-337e-4402-a5d8-8d0549e4068c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:37.436768 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 310/611] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:52:37 2022] GET /identity/v3/role_assignments?user.id=eeeab506ab5c4d6386fc899086f0abf5&scope.project.id=0ee65b9a8229495bbe97318f060d6342&role.id=d8debbb193834fac9cc25f81726f3a67 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:37.437938 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2599b9d-337e-4402-a5d8-8d0549e4068c None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:37.594278 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4e20e-c77c-4ad8-88bf-d4b21441eda4 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:37.619631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2599b9d-337e-4402-a5d8-8d0549e4068c None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:37.621100 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:37.647364 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe15da-d870-4500-9200-6c7474b726b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:37.648007 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe15da-d870-4500-9200-6c7474b726b6 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:37.648289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe15da-d870-4500-9200-6c7474b726b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:37.648556 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe15da-d870-4500-9200-6c7474b726b6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:37.649166 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe15da-d870-4500-9200-6c7474b726b6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:37.650537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe15da-d870-4500-9200-6c7474b726b6 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:37.661367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-defe15da-d870-4500-9200-6c7474b726b6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 05:52:37.661367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:37.661367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:52:37.661367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:52:37.661367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:52:37.661367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:52:37.661367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:52:37.661367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:37.661367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 05:52:37.661367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:37.661367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:52:37.661367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:37.661367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:37.661367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:37.661367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:37.661367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:37.661367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:37.662373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:37.662373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:37.662373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:37.662373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:37.662373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:37.662373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:37.662373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:52:37.662373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:52:37.662373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:52:37.662373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:52:37.662373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:37.662373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:37.662373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:37.662373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:37.662373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:37.662373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:37.663201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:37.663201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:37.663201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:37.663201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:37.663201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:37.663201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:37.663201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:37.663201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:37.663201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:37.663201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:37.663201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:52:37.663201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:37.663201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 05:52:37.663201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:37.663201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 302/612] 10.222.0.20 () {56 vars in 1129 bytes} [Thu Oct 20 05:52:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:37.664025 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe15da-d870-4500-9200-6c7474b726b6 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:37.670336 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ec0e4-ff76-4db3-85ec-c773f56db6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:37.670993 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ec0e4-ff76-4db3-85ec-c773f56db6b4 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:37.671272 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ec0e4-ff76-4db3-85ec-c773f56db6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:37.671544 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ec0e4-ff76-4db3-85ec-c773f56db6b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:37.683015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ec0e4-ff76-4db3-85ec-c773f56db6b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:37.684488 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ec0e4-ff76-4db3-85ec-c773f56db6b4 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:37.701918 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-d16ec0e4-ff76-4db3-85ec-c773f56db6b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:37.701918 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:37.701918 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:37.701918 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:37.701918 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:37.701918 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:37.701918 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:37.701918 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:37.701918 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:37.701918 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:37.701918 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:37.701918 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:37.701918 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:37.701918 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:37.701918 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:37.701918 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:37.703097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:37.703097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:37.703097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:37.703097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:37.703097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:37.703097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:37.703097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:37.703097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:37.703097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:37.703097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:37.703097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:37.703097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:37.703097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:37.703097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:37.703097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:37.703097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:37.703097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:37.704196 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:37.704196 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:37.704196 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:37.704196 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 311/613] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:37.704196 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ec0e4-ff76-4db3-85ec-c773f56db6b4 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:37.729066 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:37.777957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe15da-d870-4500-9200-6c7474b726b6 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:37.817398 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eaf508-90d9-4990-879a-78557de4304c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:37.818377 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaf508-90d9-4990-879a-78557de4304c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:37.818811 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaf508-90d9-4990-879a-78557de4304c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:37.819224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaf508-90d9-4990-879a-78557de4304c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:37.820050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eaf508-90d9-4990-879a-78557de4304c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:37.821729 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eaf508-90d9-4990-879a-78557de4304c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:37.823863 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ec0e4-ff76-4db3-85ec-c773f56db6b4 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:37.831508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-d7eaf508-90d9-4990-879a-78557de4304c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 05:52:37.831508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:37.831508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:52:37.831508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:52:37.831508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:52:37.831508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:52:37.831508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:52:37.831508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:37.831508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 05:52:37.831508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:37.831508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:52:37.831508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:37.831508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:37.831508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:37.831508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:37.831508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:37.832646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:37.832646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:37.832646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:37.832646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:37.832646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:37.832646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:37.832646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:37.832646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:52:37.832646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:52:37.832646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:52:37.832646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:52:37.832646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:37.832646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:37.832646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:37.832646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:37.832646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:37.832646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:37.833698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:37.833698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:37.833698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:37.833698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:37.833698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:37.833698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:37.833698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:37.833698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:37.833698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:37.833698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:37.833698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:52:37.833698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:37.833698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 05:52:37.833698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:37.836333 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 303/614] 10.222.0.20 () {56 vars in 1132 bytes} [Thu Oct 20 05:52:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:37.837473 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eaf508-90d9-4990-879a-78557de4304c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:37.869413 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec7b35-0478-415c-af53-5d7afaa75e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:37.869947 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec7b35-0478-415c-af53-5d7afaa75e76 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:37.870227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec7b35-0478-415c-af53-5d7afaa75e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:37.870494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec7b35-0478-415c-af53-5d7afaa75e76 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:37.871115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bec7b35-0478-415c-af53-5d7afaa75e76 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:37.871411 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bec7b35-0478-415c-af53-5d7afaa75e76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:37.872786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bec7b35-0478-415c-af53-5d7afaa75e76 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:37.882188 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 312/615] 10.222.0.20 () {56 vars in 1124 bytes} [Thu Oct 20 05:52:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:37.882878 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec7b35-0478-415c-af53-5d7afaa75e76 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:37.895235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:37.959812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eaf508-90d9-4990-879a-78557de4304c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:37.969296 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:37.985215 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec7b35-0478-415c-af53-5d7afaa75e76 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:38.000889 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab6efe6-b02f-4b1a-af3d-1117faf4e2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:38.002275 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6efe6-b02f-4b1a-af3d-1117faf4e2b6 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:38.002811 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6efe6-b02f-4b1a-af3d-1117faf4e2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:38.003347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6efe6-b02f-4b1a-af3d-1117faf4e2b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:38.004210 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab6efe6-b02f-4b1a-af3d-1117faf4e2b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:38.004726 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab6efe6-b02f-4b1a-af3d-1117faf4e2b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:38.006467 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab6efe6-b02f-4b1a-af3d-1117faf4e2b6 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:38.021158 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69640d7f-bbd3-496a-a326-651c4b3e8ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:38.021787 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69640d7f-bbd3-496a-a326-651c4b3e8ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:38.022063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69640d7f-bbd3-496a-a326-651c4b3e8ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:38.022393 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69640d7f-bbd3-496a-a326-651c4b3e8ae5 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:38.022970 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 304/616] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:38.023075 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69640d7f-bbd3-496a-a326-651c4b3e8ae5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:38.024087 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab6efe6-b02f-4b1a-af3d-1117faf4e2b6 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:38.027719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69640d7f-bbd3-496a-a326-651c4b3e8ae5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:38.029181 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69640d7f-bbd3-496a-a326-651c4b3e8ae5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:38.038720 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 313/617] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Oct 20 05:52:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:38.039597 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69640d7f-bbd3-496a-a326-651c4b3e8ae5 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:38.134725 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab6efe6-b02f-4b1a-af3d-1117faf4e2b6 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:38.145781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:38.146781 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69640d7f-bbd3-496a-a326-651c4b3e8ae5 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:38.173380 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e4be0-7bf8-48a2-b094-f26dbcec88aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:38.175240 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e4be0-7bf8-48a2-b094-f26dbcec88aa None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:38.175685 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e4be0-7bf8-48a2-b094-f26dbcec88aa None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:38.176055 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e4be0-7bf8-48a2-b094-f26dbcec88aa None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:38.176799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e4be0-7bf8-48a2-b094-f26dbcec88aa None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:38.177202 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e4be0-7bf8-48a2-b094-f26dbcec88aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:38.178870 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e4be0-7bf8-48a2-b094-f26dbcec88aa None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:38.183491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c1575-2493-4fd0-8653-3355ac6a629e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:38.184123 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c1575-2493-4fd0-8653-3355ac6a629e None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:38.184399 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c1575-2493-4fd0-8653-3355ac6a629e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:38.184669 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c1575-2493-4fd0-8653-3355ac6a629e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:38.185552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 305/618] 10.222.0.20 () {56 vars in 1124 bytes} [Thu Oct 20 05:52:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:38.186457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e4be0-7bf8-48a2-b094-f26dbcec88aa None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:38.196083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c1575-2493-4fd0-8653-3355ac6a629e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:38.197574 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c1575-2493-4fd0-8653-3355ac6a629e None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:38.209831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-a54c1575-2493-4fd0-8653-3355ac6a629e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 20 05:52:38.209831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:38.209831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:38.209831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:38.209831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:38.209831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:38.209831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:38.209831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:38.209831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:38.209831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:38.209831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:38.209831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:38.209831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:52:38.209831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:52:38.209831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:52:38.209831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:52:38.215550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:38.215550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:38.215550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:52:38.215550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:38.215550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:52:38.215550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:38.215550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:38.215550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:38.215550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:38.215550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:38.215550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:38.215550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:38.215550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:38.215550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:38.215550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:38.215550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:38.215550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:38.215550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:38.216532 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:38.216532 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:38.216532 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:52:38.216532 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:52:38.216532 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:52:38.216532 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:52:38.216532 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:52:38.216532 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:52:38.216532 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 20 05:52:38.216532 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:38.216532 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 314/619] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Oct 20 05:52:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:38.216532 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725d108-b804-4859-8819-65fe29a40ae5 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:38.216532 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725d108-b804-4859-8819-65fe29a40ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:38.217278 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725d108-b804-4859-8819-65fe29a40ae5 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:38.217278 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 315/620] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:38.217278 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725d108-b804-4859-8819-65fe29a40ae5 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:38.228929 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:38.304135 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e4be0-7bf8-48a2-b094-f26dbcec88aa None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:38.339370 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725d108-b804-4859-8819-65fe29a40ae5 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:38.340776 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23247062-8f67-4d96-9767-43b69df2632e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:38.341386 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23247062-8f67-4d96-9767-43b69df2632e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:38.341668 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23247062-8f67-4d96-9767-43b69df2632e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:38.341941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23247062-8f67-4d96-9767-43b69df2632e None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:38.342872 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23247062-8f67-4d96-9767-43b69df2632e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:38.343136 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23247062-8f67-4d96-9767-43b69df2632e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:38.344504 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23247062-8f67-4d96-9767-43b69df2632e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:38.356743 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 306/621] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Oct 20 05:52:38 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:38.356743 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23247062-8f67-4d96-9767-43b69df2632e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:38.390064 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:38.396128 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a3121-8bb2-4c78-8da5-e55a3066486e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:38.396809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a3121-8bb2-4c78-8da5-e55a3066486e None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:38.398031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a3121-8bb2-4c78-8da5-e55a3066486e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:38.398313 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a3121-8bb2-4c78-8da5-e55a3066486e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:38.399017 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a3121-8bb2-4c78-8da5-e55a3066486e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:38.400442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a3121-8bb2-4c78-8da5-e55a3066486e None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:38.431611 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 316/622] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 05:52:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:52:38.433698 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b69db9-2ca3-460d-8905-60563c77d7da None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:38.433992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b69db9-2ca3-460d-8905-60563c77d7da None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:38.434475 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b69db9-2ca3-460d-8905-60563c77d7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:38.458643 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-d4b69db9-2ca3-460d-8905-60563c77d7da None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:38.460594 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d4b69db9-2ca3-460d-8905-60563c77d7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:38.490482 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23247062-8f67-4d96-9767-43b69df2632e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:38.537155 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:38.541482 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5fb3a-d9ac-4afa-9bb8-29801a70de42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:38.542515 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5fb3a-d9ac-4afa-9bb8-29801a70de42 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:38.543080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5fb3a-d9ac-4afa-9bb8-29801a70de42 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:38.543546 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5fb3a-d9ac-4afa-9bb8-29801a70de42 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:38.545219 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5fb3a-d9ac-4afa-9bb8-29801a70de42 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:38.545793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5fb3a-d9ac-4afa-9bb8-29801a70de42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:38.548238 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 317/623] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:52:38.549435 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5fb3a-d9ac-4afa-9bb8-29801a70de42 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:38.571765 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 307/624] 10.222.0.20 () {56 vars in 1149 bytes} [Thu Oct 20 05:52:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:38.577283 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b69db9-2ca3-460d-8905-60563c77d7da None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:38.704511 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b69db9-2ca3-460d-8905-60563c77d7da None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:38.748373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4867d5-6cfd-4a50-a681-b5e0dd32012a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:38.749384 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4867d5-6cfd-4a50-a681-b5e0dd32012a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:38.749864 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4867d5-6cfd-4a50-a681-b5e0dd32012a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:38.750407 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4867d5-6cfd-4a50-a681-b5e0dd32012a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:38.833298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:38.841653 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4867d5-6cfd-4a50-a681-b5e0dd32012a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:38.843115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4867d5-6cfd-4a50-a681-b5e0dd32012a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:38.877842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5fb3a-d9ac-4afa-9bb8-29801a70de42 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:38.954810 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 318/625] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:52:38 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 05:52:38.961553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4867d5-6cfd-4a50-a681-b5e0dd32012a None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:39.062038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:39.071162 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4867d5-6cfd-4a50-a681-b5e0dd32012a None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:39.073021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:39.093744 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5fb3a-d9ac-4afa-9bb8-29801a70de42 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:39.127407 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2af7b9-10e0-4131-9857-26654664ed53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:39.128153 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2af7b9-10e0-4131-9857-26654664ed53 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:39.128439 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2af7b9-10e0-4131-9857-26654664ed53 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:39.128766 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2af7b9-10e0-4131-9857-26654664ed53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:39.140225 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2af7b9-10e0-4131-9857-26654664ed53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:39.142286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2af7b9-10e0-4131-9857-26654664ed53 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:39.161147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-9d2af7b9-10e0-4131-9857-26654664ed53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:39.161147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:39.161147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:39.161147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:39.161147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:39.161147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:39.161147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:39.161147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:39.161147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:39.161147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:39.161147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:39.161147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:39.161147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:39.161147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:39.161147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:39.161147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:39.162169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:39.162169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:39.162169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:39.162169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:39.162169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:39.162169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:39.162169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:39.162169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:39.162169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:39.162169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:39.162169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:39.162169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:39.162169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:39.162169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:39.162169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:39.162169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:39.162169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:39.163155 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:39.163155 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:39.163155 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:39.167684 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 319/626] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:52:39.169468 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2af7b9-10e0-4131-9857-26654664ed53 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:39.193948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93f944-aa73-4b62-a488-c6c994110f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:39.194597 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93f944-aa73-4b62-a488-c6c994110f25 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:39.194875 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93f944-aa73-4b62-a488-c6c994110f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:39.195143 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93f944-aa73-4b62-a488-c6c994110f25 None admin] PATH_INFO: `/v3/services/network` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:39.195722 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93f944-aa73-4b62-a488-c6c994110f25 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:39.197181 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93f944-aa73-4b62-a488-c6c994110f25 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:39.222739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-3c93f944-aa73-4b62-a488-c6c994110f25 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 05:52:39.222739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:39.222739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:52:39.222739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:52:39.222739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:52:39.222739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:52:39.222739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:52:39.222739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:39.222739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 05:52:39.222739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:39.222739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:52:39.222739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:39.222739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:39.222739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:39.222739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:39.222739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:39.222739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:39.223764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:39.223764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:39.223764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:39.223764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:39.223764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:39.223764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:39.223764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:52:39.223764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:52:39.223764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:52:39.223764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:52:39.223764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:39.223764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:39.223764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:39.223764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:39.223764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:39.223764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:39.224624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:39.224624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:39.224624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:39.224624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:39.224624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:39.224624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:39.224624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:39.224624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:39.224624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:39.224624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:39.224624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:52:39.224624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:39.224624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 05:52:39.224624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:39.224624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 308/627] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:38 2022] GET /identity/v3/services/network => generated 88 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:39.233637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93f944-aa73-4b62-a488-c6c994110f25 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:39.276672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2af7b9-10e0-4131-9857-26654664ed53 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:39.322785 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79a253-dae2-4d2b-b817-0822dc48662c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:39.323391 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79a253-dae2-4d2b-b817-0822dc48662c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:39.323672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79a253-dae2-4d2b-b817-0822dc48662c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:39.323946 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79a253-dae2-4d2b-b817-0822dc48662c None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:39.324598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79a253-dae2-4d2b-b817-0822dc48662c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:39.325019 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79a253-dae2-4d2b-b817-0822dc48662c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:39.327110 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79a253-dae2-4d2b-b817-0822dc48662c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:39.345827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 320/628] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:39.347236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79a253-dae2-4d2b-b817-0822dc48662c None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:39.369097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:39.502528 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93f944-aa73-4b62-a488-c6c994110f25 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:39.520866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79a253-dae2-4d2b-b817-0822dc48662c None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:39.550403 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683091f-27bb-4dc5-80da-aaac9ed889d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:39.553472 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683091f-27bb-4dc5-80da-aaac9ed889d1 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:39.554141 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683091f-27bb-4dc5-80da-aaac9ed889d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:39.557378 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683091f-27bb-4dc5-80da-aaac9ed889d1 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:39.558388 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683091f-27bb-4dc5-80da-aaac9ed889d1 None admin] RBAC: Adding query filter params (name=network) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:39.561084 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683091f-27bb-4dc5-80da-aaac9ed889d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:39.562782 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683091f-27bb-4dc5-80da-aaac9ed889d1 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:39.576827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 309/629] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Oct 20 05:52:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:39.582869 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683091f-27bb-4dc5-80da-aaac9ed889d1 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:39.615610 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49e168-c360-4766-811d-89b594cc41c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:39.617657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49e168-c360-4766-811d-89b594cc41c5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:39.618089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49e168-c360-4766-811d-89b594cc41c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:39.618505 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49e168-c360-4766-811d-89b594cc41c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:39.619569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:39.636329 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a49e168-c360-4766-811d-89b594cc41c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:39.639120 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a49e168-c360-4766-811d-89b594cc41c5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:39.659139 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-1a49e168-c360-4766-811d-89b594cc41c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:39.659139 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:39.659139 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:39.659139 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:39.659139 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:39.659139 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:39.659139 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:39.659139 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:39.659139 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:39.659139 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:39.659139 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:39.659139 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:39.659139 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:52:39.659139 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:52:39.659139 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:52:39.660922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:52:39.660922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:39.660922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:39.660922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:39.660922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:39.660922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:39.660922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:39.660922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:39.660922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:39.660922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:39.660922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:39.660922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:39.660922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:39.660922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:39.660922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:39.660922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:39.660922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:39.662388 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:52:39.662388 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:52:39.662388 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:52:39.662388 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:52:39.662388 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:52:39.662388 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:52:39.662388 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:39.662388 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:39.668815 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 321/630] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:39.671023 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49e168-c360-4766-811d-89b594cc41c5 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:39.755089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:39.832237 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683091f-27bb-4dc5-80da-aaac9ed889d1 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:39.842602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49e168-c360-4766-811d-89b594cc41c5 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:39.913378 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51381c54-2589-4239-8a4c-62ca4ffb8a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:39.914224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51381c54-2589-4239-8a4c-62ca4ffb8a5c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:39.914310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51381c54-2589-4239-8a4c-62ca4ffb8a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:39.914544 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51381c54-2589-4239-8a4c-62ca4ffb8a5c None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:39.915141 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51381c54-2589-4239-8a4c-62ca4ffb8a5c None admin] RBAC: Adding query filter params (type=network) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:39.915443 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51381c54-2589-4239-8a4c-62ca4ffb8a5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:39.916840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51381c54-2589-4239-8a4c-62ca4ffb8a5c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:39.931244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 310/631] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Oct 20 05:52:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:39.933819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622e21b-bd83-45eb-9d4f-1f6b890a1af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:39.934597 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622e21b-bd83-45eb-9d4f-1f6b890a1af0 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:39.934918 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622e21b-bd83-45eb-9d4f-1f6b890a1af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:39.935191 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622e21b-bd83-45eb-9d4f-1f6b890a1af0 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:39.936789 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3c805-ade7-4b37-b37d-c99c8b8e2cb0 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:39.937073 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3c805-ade7-4b37-b37d-c99c8b8e2cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:39.937343 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3c805-ade7-4b37-b37d-c99c8b8e2cb0 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:39.938108 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1622e21b-bd83-45eb-9d4f-1f6b890a1af0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:39.938655 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1622e21b-bd83-45eb-9d4f-1f6b890a1af0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:39.938982 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 311/632] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:39.939806 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3c805-ade7-4b37-b37d-c99c8b8e2cb0 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:39.940990 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1622e21b-bd83-45eb-9d4f-1f6b890a1af0 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:39.972872 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 322/633] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Oct 20 05:52:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:39.975191 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a921e-8546-4a55-b400-5ee40f555296 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:39.975738 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a921e-8546-4a55-b400-5ee40f555296 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:39.976170 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a921e-8546-4a55-b400-5ee40f555296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:40.001502 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-b60a921e-8546-4a55-b400-5ee40f555296 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:40.002972 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b60a921e-8546-4a55-b400-5ee40f555296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:40.046730 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:40.073145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3c805-ade7-4b37-b37d-c99c8b8e2cb0 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:40.084409 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 323/634] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:52:40.085963 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a921e-8546-4a55-b400-5ee40f555296 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:40.129491 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ad7ab-8660-4d93-809d-39c5a7d64ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:40.130149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ad7ab-8660-4d93-809d-39c5a7d64ffb None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:40.130435 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ad7ab-8660-4d93-809d-39c5a7d64ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:40.130749 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ad7ab-8660-4d93-809d-39c5a7d64ffb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:40.132229 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ad7ab-8660-4d93-809d-39c5a7d64ffb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:40.133696 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ad7ab-8660-4d93-809d-39c5a7d64ffb None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:40.181347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 312/635] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 05:52:39 2022] POST /identity/v3/endpoints => generated 337 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:52:40.221980 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:40.247081 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a921e-8546-4a55-b400-5ee40f555296 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:40.313617 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebb57a-b7d0-4ab1-b509-32ae5ac2276d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:40.314362 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebb57a-b7d0-4ab1-b509-32ae5ac2276d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:40.315155 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebb57a-b7d0-4ab1-b509-32ae5ac2276d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:40.315429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebb57a-b7d0-4ab1-b509-32ae5ac2276d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:40.316312 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beebb57a-b7d0-4ab1-b509-32ae5ac2276d None admin] RBAC: Adding query filter params (role.id=afd859d2deed4d51aa3f6326920441c8, scope.project.id=0ee65b9a8229495bbe97318f060d6342, user.id=7a59f547a20e48ee88e3e620ba1e87f1) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:40.316625 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beebb57a-b7d0-4ab1-b509-32ae5ac2276d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:40.318128 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beebb57a-b7d0-4ab1-b509-32ae5ac2276d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:40.329449 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 324/636] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:52:40 2022] GET /identity/v3/role_assignments?user.id=7a59f547a20e48ee88e3e620ba1e87f1&scope.project.id=0ee65b9a8229495bbe97318f060d6342&role.id=afd859d2deed4d51aa3f6326920441c8 => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:40.371232 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ad7ab-8660-4d93-809d-39c5a7d64ffb None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:40.506251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ad7ab-8660-4d93-809d-39c5a7d64ffb None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:40.544490 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df746c14-a415-4e4c-be49-b05eb71210f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:40.545046 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df746c14-a415-4e4c-be49-b05eb71210f6 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:40.545326 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df746c14-a415-4e4c-be49-b05eb71210f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:40.545590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df746c14-a415-4e4c-be49-b05eb71210f6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:40.546238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df746c14-a415-4e4c-be49-b05eb71210f6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:40.547623 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df746c14-a415-4e4c-be49-b05eb71210f6 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:40.549667 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:40.558508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-df746c14-a415-4e4c-be49-b05eb71210f6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 05:52:40.558508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:40.558508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:52:40.558508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:52:40.558508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:52:40.558508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:52:40.558508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:52:40.558508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:40.558508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 05:52:40.558508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:40.558508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:52:40.558508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:40.558508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:40.558508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:40.558508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:40.558508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:40.558508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:40.559507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:40.559507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:40.559507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:40.559507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:40.559507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:40.559507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:40.559507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:52:40.559507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:52:40.559507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:52:40.559507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:52:40.559507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:40.559507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:40.559507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:40.559507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:40.559507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:40.559507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:40.560513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:40.560513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:40.560513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:40.560513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:40.560513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:40.560513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:40.560513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:40.560513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:40.560513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:40.560513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:40.560513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:52:40.560513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:40.560513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 05:52:40.560513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:40.560513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 313/637] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Oct 20 05:52:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:40.566321 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebb57a-b7d0-4ab1-b509-32ae5ac2276d None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:40.673676 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebb57a-b7d0-4ab1-b509-32ae5ac2276d None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:40.692108 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:40.712788 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505e944-6a05-4c3f-bde6-d0bdc920b0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:40.713399 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505e944-6a05-4c3f-bde6-d0bdc920b0c7 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:40.713672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505e944-6a05-4c3f-bde6-d0bdc920b0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:40.713943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505e944-6a05-4c3f-bde6-d0bdc920b0c7 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:40.714566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505e944-6a05-4c3f-bde6-d0bdc920b0c7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:40.714864 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505e944-6a05-4c3f-bde6-d0bdc920b0c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:40.716225 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505e944-6a05-4c3f-bde6-d0bdc920b0c7 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:40.721921 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 325/638] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Oct 20 05:52:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:40.727406 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df746c14-a415-4e4c-be49-b05eb71210f6 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:40.842591 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df746c14-a415-4e4c-be49-b05eb71210f6 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:40.880199 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbb390-3756-4bdc-9368-6e6d8a17aa36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:40.880850 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbb390-3756-4bdc-9368-6e6d8a17aa36 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:40.881134 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbb390-3756-4bdc-9368-6e6d8a17aa36 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:40.881407 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbb390-3756-4bdc-9368-6e6d8a17aa36 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:40.882041 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbb390-3756-4bdc-9368-6e6d8a17aa36 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:40.882343 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbb390-3756-4bdc-9368-6e6d8a17aa36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:40.883720 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbb390-3756-4bdc-9368-6e6d8a17aa36 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:40.889107 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 314/639] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Oct 20 05:52:40 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:41.107948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d9f6f-2b79-4807-b33a-ed964e0cf5aa None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:41.108291 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d9f6f-2b79-4807-b33a-ed964e0cf5aa None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:41.108468 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d9f6f-2b79-4807-b33a-ed964e0cf5aa None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:41.110075 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 326/640] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:41.115486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f9e08-6ae3-4102-9b30-d78c1d8af89e None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:41.115947 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f9e08-6ae3-4102-9b30-d78c1d8af89e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:41.116046 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f9e08-6ae3-4102-9b30-d78c1d8af89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:41.134497 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-e49f9e08-6ae3-4102-9b30-d78c1d8af89e None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:41.135943 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e49f9e08-6ae3-4102-9b30-d78c1d8af89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:41.196916 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:41.209164 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 315/641] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:52:41.345810 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c657a5-4a38-4954-9d2d-1f80651a1410 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:41.346486 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c657a5-4a38-4954-9d2d-1f80651a1410 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:41.346983 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c657a5-4a38-4954-9d2d-1f80651a1410 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:41.348752 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 327/642] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:41.354294 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022440cf-53c6-4cbe-80bf-dbf5cb3f82b9 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:41.354810 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022440cf-53c6-4cbe-80bf-dbf5cb3f82b9 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:41.355276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022440cf-53c6-4cbe-80bf-dbf5cb3f82b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:41.375342 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-022440cf-53c6-4cbe-80bf-dbf5cb3f82b9 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:41.377438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-022440cf-53c6-4cbe-80bf-dbf5cb3f82b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:41.451339 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 316/643] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:52:41.497812 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c657a5-4a38-4954-9d2d-1f80651a1410 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:41.607078 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c657a5-4a38-4954-9d2d-1f80651a1410 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:41.644844 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2333aea-931e-4ed1-bf75-13c4542c8e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:41.645467 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2333aea-931e-4ed1-bf75-13c4542c8e92 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:41.645747 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2333aea-931e-4ed1-bf75-13c4542c8e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:41.646017 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2333aea-931e-4ed1-bf75-13c4542c8e92 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:41.646663 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2333aea-931e-4ed1-bf75-13c4542c8e92 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:41.648048 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2333aea-931e-4ed1-bf75-13c4542c8e92 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:41.666377 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 328/644] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Oct 20 05:52:41 2022] POST /identity/v3/services => generated 269 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:52:41.724919 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022440cf-53c6-4cbe-80bf-dbf5cb3f82b9 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:41.838372 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022440cf-53c6-4cbe-80bf-dbf5cb3f82b9 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:41.865637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:41.887467 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3166c-a547-408c-9b98-434de20a91c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:41.888106 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3166c-a547-408c-9b98-434de20a91c4 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:41.888382 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3166c-a547-408c-9b98-434de20a91c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:41.888650 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3166c-a547-408c-9b98-434de20a91c4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:41.889260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3166c-a547-408c-9b98-434de20a91c4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:41.890613 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3166c-a547-408c-9b98-434de20a91c4 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:41.901017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-94d3166c-a547-408c-9b98-434de20a91c4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 05:52:41.901017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:41.901017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:52:41.901017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:52:41.901017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:52:41.901017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:52:41.901017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:52:41.901017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:41.901017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 05:52:41.901017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:41.901017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:52:41.901017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:41.901017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:41.901017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:41.901017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:41.901017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:41.901017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:41.901987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:41.901987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:41.901987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:41.901987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:41.901987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:41.901987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:41.901987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:52:41.901987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:52:41.901987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:52:41.901987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:52:41.901987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:41.901987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:41.901987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:41.901987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:41.901987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:41.901987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:41.902831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:41.902831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:41.902831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:41.902831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:41.902831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:41.902831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:41.902831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:41.902831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:41.902831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:41.902831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:41.902831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:52:41.902831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:41.902831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 05:52:41.902831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:41.902831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 317/645] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Oct 20 05:52:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:41.910525 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2333aea-931e-4ed1-bf75-13c4542c8e92 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:41.961235 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:42.022986 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2333aea-931e-4ed1-bf75-13c4542c8e92 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:42.056947 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2edb712-b0f0-4f6c-97ec-821c55c291f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:42.057558 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edb712-b0f0-4f6c-97ec-821c55c291f4 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:42.057830 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edb712-b0f0-4f6c-97ec-821c55c291f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:42.058097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edb712-b0f0-4f6c-97ec-821c55c291f4 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:42.058721 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2edb712-b0f0-4f6c-97ec-821c55c291f4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:42.059017 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2edb712-b0f0-4f6c-97ec-821c55c291f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:42.060377 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2edb712-b0f0-4f6c-97ec-821c55c291f4 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:42.065883 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 329/646] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Oct 20 05:52:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:42.071586 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3166c-a547-408c-9b98-434de20a91c4 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:42.176224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3166c-a547-408c-9b98-434de20a91c4 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:42.212557 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43579f0d-771b-4994-b3e8-9b0fb61dd13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:42.213570 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43579f0d-771b-4994-b3e8-9b0fb61dd13c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:42.214060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43579f0d-771b-4994-b3e8-9b0fb61dd13c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:42.214519 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43579f0d-771b-4994-b3e8-9b0fb61dd13c None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:42.215328 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43579f0d-771b-4994-b3e8-9b0fb61dd13c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:42.215828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43579f0d-771b-4994-b3e8-9b0fb61dd13c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:42.217400 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43579f0d-771b-4994-b3e8-9b0fb61dd13c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:42.223261 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 318/647] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Oct 20 05:52:42 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:42.812146 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cf80f-fa90-444a-9093-1620b7abe8a6 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:42.812674 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cf80f-fa90-444a-9093-1620b7abe8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:42.812674 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cf80f-fa90-444a-9093-1620b7abe8a6 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:42.814799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 330/648] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:42.820091 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70918cd7-9dea-44ee-a459-74f9ee9dd47b None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:42.820380 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70918cd7-9dea-44ee-a459-74f9ee9dd47b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:42.820658 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70918cd7-9dea-44ee-a459-74f9ee9dd47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:42.839232 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-70918cd7-9dea-44ee-a459-74f9ee9dd47b None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:42.840420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-70918cd7-9dea-44ee-a459-74f9ee9dd47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:42.889110 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:42.918841 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 319/649] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:42 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:52:43.094810 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe308b7-0840-4068-a827-cefe8bff1a2e None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:43.095114 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe308b7-0840-4068-a827-cefe8bff1a2e None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:43.095296 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe308b7-0840-4068-a827-cefe8bff1a2e None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:43.097259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 331/650] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:43.102705 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07719ea8-e1d5-4726-9265-059757a4448d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:43.103213 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07719ea8-e1d5-4726-9265-059757a4448d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:43.103682 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07719ea8-e1d5-4726-9265-059757a4448d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:43.123287 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-07719ea8-e1d5-4726-9265-059757a4448d None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:43.124925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-07719ea8-e1d5-4726-9265-059757a4448d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:43.195699 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe308b7-0840-4068-a827-cefe8bff1a2e None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:43.199065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 320/651] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:43 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:52:43.306355 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe308b7-0840-4068-a827-cefe8bff1a2e None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:43.308938 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:43.343516 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0303e-39f1-49f4-8e26-64c5dabc412f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:43.344140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0303e-39f1-49f4-8e26-64c5dabc412f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:43.344414 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0303e-39f1-49f4-8e26-64c5dabc412f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:43.344682 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0303e-39f1-49f4-8e26-64c5dabc412f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:43.355882 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b0303e-39f1-49f4-8e26-64c5dabc412f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:52:43.360485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-d6b0303e-39f1-49f4-8e26-64c5dabc412f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 20 05:52:43.360485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:43.360485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:43.360485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:43.360485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:43.360485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:43.360485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:43.360485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:43.360485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:43.360485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:43.360485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:43.360485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:43.360485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:52:43.360485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:52:43.360485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:52:43.360485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:52:43.361582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:52:43.361582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:52:43.361582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:52:43.361582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:52:43.361582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:43.361582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:43.361582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:43.361582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:43.361582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:43.361582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:43.361582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:43.361582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:43.361582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:43.361582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:43.361582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:43.361582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:43.362794 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:43.362794 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:43.362794 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:43.362794 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:43.362794 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:52:43.362794 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:52:43.362794 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:52:43.362794 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:52:43.362794 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:52:43.362794 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:52:43.362794 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 20 05:52:43.362794 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:43.362794 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 332/652] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Oct 20 05:52:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:43.367532 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07719ea8-e1d5-4726-9265-059757a4448d None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:43.471568 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07719ea8-e1d5-4726-9265-059757a4448d None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:43.473062 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0303e-39f1-49f4-8e26-64c5dabc412f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:43.479785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:43.514659 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476f4cd-0137-480c-86c4-b66f189c1973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:43.515282 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476f4cd-0137-480c-86c4-b66f189c1973 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:43.515563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476f4cd-0137-480c-86c4-b66f189c1973 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:43.515834 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476f4cd-0137-480c-86c4-b66f189c1973 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:43.516500 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476f4cd-0137-480c-86c4-b66f189c1973 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:43.516836 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476f4cd-0137-480c-86c4-b66f189c1973 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:43.518773 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476f4cd-0137-480c-86c4-b66f189c1973 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:43.531973 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 321/653] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Oct 20 05:52:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:43.537750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476f4cd-0137-480c-86c4-b66f189c1973 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:43.590711 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0303e-39f1-49f4-8e26-64c5dabc412f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:43.630183 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31098792-bd9d-4e59-8a50-84ee318041ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:43.630806 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31098792-bd9d-4e59-8a50-84ee318041ae None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:43.631092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31098792-bd9d-4e59-8a50-84ee318041ae None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:43.631366 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31098792-bd9d-4e59-8a50-84ee318041ae None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:43.632807 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31098792-bd9d-4e59-8a50-84ee318041ae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:43.633523 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31098792-bd9d-4e59-8a50-84ee318041ae None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:43.654120 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 333/654] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Oct 20 05:52:43 2022] POST /identity/v3/services => generated 241 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:52:43.671010 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476f4cd-0137-480c-86c4-b66f189c1973 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:43.721938 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d50be0-5539-4717-97ac-e98974d139cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:43.723022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d50be0-5539-4717-97ac-e98974d139cd None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:43.723487 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d50be0-5539-4717-97ac-e98974d139cd None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:43.723931 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d50be0-5539-4717-97ac-e98974d139cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:43.737144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d50be0-5539-4717-97ac-e98974d139cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:43.739079 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d50be0-5539-4717-97ac-e98974d139cd None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:43.756474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-16d50be0-5539-4717-97ac-e98974d139cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:43.756474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:43.756474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:43.756474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:43.756474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:43.756474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:43.756474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:43.756474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:43.756474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:43.756474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:43.756474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:43.756474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:43.756474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:43.756474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:43.756474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:43.756474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:43.757593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:43.757593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:43.757593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:43.757593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:43.757593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:43.757593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:43.757593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:43.757593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:43.757593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:43.757593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:43.757593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:43.757593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:43.757593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:43.757593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:43.757593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:43.757593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:43.757593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:43.758492 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:43.758492 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:43.758492 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:43.760770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 322/655] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:43.770129 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31098792-bd9d-4e59-8a50-84ee318041ae None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:43.802199 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:43.892305 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31098792-bd9d-4e59-8a50-84ee318041ae None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:43.937334 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733b3cb-d8e2-4c3e-a12e-05912f0c8fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:43.938032 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733b3cb-d8e2-4c3e-a12e-05912f0c8fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:43.938309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733b3cb-d8e2-4c3e-a12e-05912f0c8fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:43.938590 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733b3cb-d8e2-4c3e-a12e-05912f0c8fc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:43.939324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733b3cb-d8e2-4c3e-a12e-05912f0c8fc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:43.939627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733b3cb-d8e2-4c3e-a12e-05912f0c8fc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:43.941235 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733b3cb-d8e2-4c3e-a12e-05912f0c8fc5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:43.956324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 334/656] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:43.961853 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d50be0-5539-4717-97ac-e98974d139cd None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:43.981029 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:44.052369 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a881631-95be-4b51-8760-58653d77c621 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:44.052835 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a881631-95be-4b51-8760-58653d77c621 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:44.053215 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a881631-95be-4b51-8760-58653d77c621 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:44.054896 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 335/657] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:44.059976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98e1c4-4a38-4f68-99e7-1791ccc3ce78 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:44.060431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98e1c4-4a38-4f68-99e7-1791ccc3ce78 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:44.060881 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98e1c4-4a38-4f68-99e7-1791ccc3ce78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:44.070764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d50be0-5539-4717-97ac-e98974d139cd None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:44.081087 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-fa98e1c4-4a38-4f68-99e7-1791ccc3ce78 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:44.082694 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fa98e1c4-4a38-4f68-99e7-1791ccc3ce78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:44.110201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0609f46-48ca-4dcb-be95-0d9cdfa5148c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:44.110828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0609f46-48ca-4dcb-be95-0d9cdfa5148c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:44.111623 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0609f46-48ca-4dcb-be95-0d9cdfa5148c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:44.112310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0609f46-48ca-4dcb-be95-0d9cdfa5148c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:44.122302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0609f46-48ca-4dcb-be95-0d9cdfa5148c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:44.124083 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0609f46-48ca-4dcb-be95-0d9cdfa5148c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:44.137641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-f0609f46-48ca-4dcb-be95-0d9cdfa5148c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 20 05:52:44.137641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:44.137641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:44.137641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:44.137641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:44.137641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:44.137641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:44.137641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:44.137641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:44.137641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:44.137641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:44.137641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:44.137641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:52:44.137641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:52:44.137641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:52:44.137641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:52:44.138840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:44.138840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:44.138840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:52:44.138840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:44.138840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:52:44.138840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:44.138840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:44.138840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:44.138840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:44.138840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:44.138840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:44.138840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:44.138840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:44.138840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:44.138840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:44.138840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:44.138840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:44.138840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:44.140194 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:44.140194 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:44.140194 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:52:44.140194 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:52:44.140194 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:52:44.140194 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:52:44.140194 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:52:44.140194 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:52:44.140194 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 20 05:52:44.140194 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:44.140194 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 323/658] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Oct 20 05:52:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:44.145573 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0609f46-48ca-4dcb-be95-0d9cdfa5148c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:44.171831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 336/659] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:44 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 05:52:44.233308 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:44.260605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0609f46-48ca-4dcb-be95-0d9cdfa5148c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:44.299822 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a68793-994d-4fba-af22-7a8440bd5c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:44.300507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a68793-994d-4fba-af22-7a8440bd5c96 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:44.300679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a68793-994d-4fba-af22-7a8440bd5c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:44.302247 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a68793-994d-4fba-af22-7a8440bd5c96 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:44.302247 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a68793-994d-4fba-af22-7a8440bd5c96 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:44.302247 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a68793-994d-4fba-af22-7a8440bd5c96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:44.305165 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a68793-994d-4fba-af22-7a8440bd5c96 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:44.326204 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 324/660] 10.222.0.20 () {56 vars in 1149 bytes} [Thu Oct 20 05:52:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:44.337860 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98e1c4-4a38-4f68-99e7-1791ccc3ce78 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:44.354661 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:44.453270 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98e1c4-4a38-4f68-99e7-1791ccc3ce78 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:44.494958 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c161285-01ca-4978-a21f-a8ff31c3169f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:44.495870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c161285-01ca-4978-a21f-a8ff31c3169f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:44.496292 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c161285-01ca-4978-a21f-a8ff31c3169f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:44.496724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c161285-01ca-4978-a21f-a8ff31c3169f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:44.508570 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c161285-01ca-4978-a21f-a8ff31c3169f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:44.510500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c161285-01ca-4978-a21f-a8ff31c3169f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:44.519085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a68793-994d-4fba-af22-7a8440bd5c96 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:44.526512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-3c161285-01ca-4978-a21f-a8ff31c3169f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:44.526512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:44.526512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:44.526512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:44.526512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:44.526512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:44.526512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:44.526512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:44.526512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:44.526512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:44.526512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:44.526512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:44.526512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:44.526512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:44.526512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:44.526512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:44.527654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:44.527654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:44.527654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:44.527654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:44.527654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:44.527654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:44.527654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:44.527654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:44.527654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:44.527654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:44.527654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:44.527654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:44.527654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:44.527654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:44.527654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:44.527654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:44.527654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:44.528745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:44.528745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:44.528745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:44.530486 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 337/661] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:44.537096 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c161285-01ca-4978-a21f-a8ff31c3169f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:44.609534 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:44.634785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a68793-994d-4fba-af22-7a8440bd5c96 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:44.648410 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c161285-01ca-4978-a21f-a8ff31c3169f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:44.676124 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288046f-a1f7-4856-b2af-263a34416439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:44.676881 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288046f-a1f7-4856-b2af-263a34416439 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:44.677056 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288046f-a1f7-4856-b2af-263a34416439 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:44.677327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288046f-a1f7-4856-b2af-263a34416439 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:44.677946 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d288046f-a1f7-4856-b2af-263a34416439 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:44.679281 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d288046f-a1f7-4856-b2af-263a34416439 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:44.681097 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:44.685292 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6eec11-9d31-4146-9c97-9b752bd861f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:44.685897 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6eec11-9d31-4146-9c97-9b752bd861f7 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:44.686176 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6eec11-9d31-4146-9c97-9b752bd861f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:44.686445 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6eec11-9d31-4146-9c97-9b752bd861f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:44.687101 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6eec11-9d31-4146-9c97-9b752bd861f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:44.687403 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6eec11-9d31-4146-9c97-9b752bd861f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:44.688756 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6eec11-9d31-4146-9c97-9b752bd861f7 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:44.689807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-d288046f-a1f7-4856-b2af-263a34416439 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 05:52:44.689807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:44.689807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:52:44.689807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:52:44.689807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:52:44.689807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:52:44.689807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:52:44.689807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:44.689807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 05:52:44.689807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:44.689807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:52:44.689807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:44.689807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:44.689807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:44.689807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:44.689807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:44.690818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:44.690818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:44.690818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:44.690818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:44.690818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:44.690818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:44.690818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:44.690818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:52:44.690818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:52:44.690818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:52:44.690818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:52:44.690818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:44.690818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:44.690818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:44.690818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:44.690818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:44.690818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:44.691739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:44.691739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:44.691739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:44.691739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:44.691739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:44.691739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:44.691739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:44.691739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:44.691739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:44.691739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:44.691739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:52:44.691739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:44.691739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 05:52:44.691739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:44.699211 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 325/662] 10.222.0.20 () {56 vars in 1132 bytes} [Thu Oct 20 05:52:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:44.699211 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0eb05-b6a7-4b26-98f7-58381ed6c5b9 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:44.699211 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0eb05-b6a7-4b26-98f7-58381ed6c5b9 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:44.699211 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0eb05-b6a7-4b26-98f7-58381ed6c5b9 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:44.700823 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 326/663] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:44.701860 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f0eb05-b6a7-4b26-98f7-58381ed6c5b9 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:44.705299 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 338/664] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:44.711090 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b530d87-8c6f-406a-8390-3245b6aedb11 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:44.711380 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b530d87-8c6f-406a-8390-3245b6aedb11 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:44.711657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b530d87-8c6f-406a-8390-3245b6aedb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:44.729794 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-6b530d87-8c6f-406a-8390-3245b6aedb11 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:44.730978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6b530d87-8c6f-406a-8390-3245b6aedb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:44.805635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 339/665] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:44 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:52:44.806871 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b530d87-8c6f-406a-8390-3245b6aedb11 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:44.810373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f0eb05-b6a7-4b26-98f7-58381ed6c5b9 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:44.846581 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3c9f0-4502-490a-8799-f107ad7ea8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:44.847194 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3c9f0-4502-490a-8799-f107ad7ea8d1 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:44.847476 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3c9f0-4502-490a-8799-f107ad7ea8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:44.847746 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3c9f0-4502-490a-8799-f107ad7ea8d1 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:44.848360 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3c9f0-4502-490a-8799-f107ad7ea8d1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:44.848676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3c9f0-4502-490a-8799-f107ad7ea8d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:44.850057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3c9f0-4502-490a-8799-f107ad7ea8d1 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:44.854857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 327/666] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Oct 20 05:52:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:44.860424 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3c9f0-4502-490a-8799-f107ad7ea8d1 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:44.885704 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:44.926603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:44.947737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b530d87-8c6f-406a-8390-3245b6aedb11 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:44.970271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3c9f0-4502-490a-8799-f107ad7ea8d1 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:44.996785 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06813f9b-c949-4776-841b-52653eb0fd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:44.998123 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06813f9b-c949-4776-841b-52653eb0fd3c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:44.999004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06813f9b-c949-4776-841b-52653eb0fd3c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:44.999757 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06813f9b-c949-4776-841b-52653eb0fd3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:45.006579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465f8759-51e2-4998-b3bc-5e6f8fd0e205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:45.007244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465f8759-51e2-4998-b3bc-5e6f8fd0e205 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:45.007524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465f8759-51e2-4998-b3bc-5e6f8fd0e205 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:45.007841 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465f8759-51e2-4998-b3bc-5e6f8fd0e205 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:45.008459 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465f8759-51e2-4998-b3bc-5e6f8fd0e205 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:45.008857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465f8759-51e2-4998-b3bc-5e6f8fd0e205 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:45.009534 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06813f9b-c949-4776-841b-52653eb0fd3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:45.010786 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465f8759-51e2-4998-b3bc-5e6f8fd0e205 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:45.011655 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06813f9b-c949-4776-841b-52653eb0fd3c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:45.021389 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 328/667] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Oct 20 05:52:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:45.022849 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465f8759-51e2-4998-b3bc-5e6f8fd0e205 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:45.026100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-06813f9b-c949-4776-841b-52653eb0fd3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:45.026100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:45.026100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:45.026100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:45.026100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:45.026100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:45.026100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:45.026100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:45.026100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:45.026100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:45.026100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:45.026100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:45.026100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:52:45.026100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:52:45.026100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:52:45.026978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:52:45.026978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:45.026978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:45.026978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:45.026978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:45.026978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:45.026978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:45.026978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:45.026978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:45.026978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:45.026978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:45.026978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:45.026978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:45.026978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:45.026978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:45.026978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:45.026978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:45.027906 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:52:45.027906 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:52:45.027906 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:52:45.027906 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:52:45.027906 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:52:45.027906 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:52:45.027906 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:45.027906 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:45.030019 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 340/668] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:45.037290 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06813f9b-c949-4776-841b-52653eb0fd3c None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:45.138029 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465f8759-51e2-4998-b3bc-5e6f8fd0e205 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:45.162186 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06813f9b-c949-4776-841b-52653eb0fd3c None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:45.164258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:45.174092 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3ac9b-d7f9-4afd-9b55-84662b59c6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:45.175005 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3ac9b-d7f9-4afd-9b55-84662b59c6e5 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:45.175289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3ac9b-d7f9-4afd-9b55-84662b59c6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:45.175564 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3ac9b-d7f9-4afd-9b55-84662b59c6e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:45.176259 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c3ac9b-d7f9-4afd-9b55-84662b59c6e5 None admin] RBAC: Adding query filter params (interface=public, service_id=0cdf35d64e114224b9e51085690f70ec, region_id=RegionOne) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:45.176565 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c3ac9b-d7f9-4afd-9b55-84662b59c6e5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:45.177959 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c3ac9b-d7f9-4afd-9b55-84662b59c6e5 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:45.182054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:45.184080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 329/669] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Oct 20 05:52:45 2022] GET /identity/v3/endpoints?service_id=0cdf35d64e114224b9e51085690f70ec&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:45.184862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3ac9b-d7f9-4afd-9b55-84662b59c6e5 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:45.201732 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1435a81-ea17-4b63-ba15-0778bc489fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:45.202420 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1435a81-ea17-4b63-ba15-0778bc489fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:45.202752 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1435a81-ea17-4b63-ba15-0778bc489fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:45.203026 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1435a81-ea17-4b63-ba15-0778bc489fc5 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:45.203826 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1435a81-ea17-4b63-ba15-0778bc489fc5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:45.204178 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1435a81-ea17-4b63-ba15-0778bc489fc5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:45.205819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1435a81-ea17-4b63-ba15-0778bc489fc5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:45.225802 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 341/670] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Oct 20 05:52:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:45.225802 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1435a81-ea17-4b63-ba15-0778bc489fc5 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:45.294464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3ac9b-d7f9-4afd-9b55-84662b59c6e5 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:45.331728 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765d713-bcf1-46ed-afad-415303f2016a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:45.332340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765d713-bcf1-46ed-afad-415303f2016a None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:45.332639 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765d713-bcf1-46ed-afad-415303f2016a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:45.332928 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765d713-bcf1-46ed-afad-415303f2016a None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:45.333569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765d713-bcf1-46ed-afad-415303f2016a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:45.334969 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765d713-bcf1-46ed-afad-415303f2016a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:45.339971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1435a81-ea17-4b63-ba15-0778bc489fc5 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:45.351581 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 330/671] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Oct 20 05:52:45 2022] POST /identity/v3/services => generated 251 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:52:45.352655 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765d713-bcf1-46ed-afad-415303f2016a None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:45.392815 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e285313-fa1f-4298-b43f-43ab39ccd105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:45.393466 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e285313-fa1f-4298-b43f-43ab39ccd105 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:45.393734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e285313-fa1f-4298-b43f-43ab39ccd105 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:45.394008 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e285313-fa1f-4298-b43f-43ab39ccd105 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:45.394628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e285313-fa1f-4298-b43f-43ab39ccd105 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:45.396085 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e285313-fa1f-4298-b43f-43ab39ccd105 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:45.403121 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 342/672] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Oct 20 05:52:45 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 05:52:45.423155 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:45.491904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765d713-bcf1-46ed-afad-415303f2016a None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:45.535373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b8793-d8b9-4a84-8da2-9c56e6c258af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:45.536554 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b8793-d8b9-4a84-8da2-9c56e6c258af None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:45.537077 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b8793-d8b9-4a84-8da2-9c56e6c258af None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:45.537616 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b8793-d8b9-4a84-8da2-9c56e6c258af None admin] PATH_INFO: `/v3/projects/0ee65b9a8229495bbe97318f060d6342/users/7a59f547a20e48ee88e3e620ba1e87f1/roles/b53e713d4b4a460b9465321b090b0455` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:45.540374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b8793-d8b9-4a84-8da2-9c56e6c258af None admin] RBAC: Authorizing `identity:create_grant(project_id=0ee65b9a8229495bbe97318f060d6342, user_id=7a59f547a20e48ee88e3e620ba1e87f1, role_id=b53e713d4b4a460b9465321b090b0455)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:45.542234 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b8793-d8b9-4a84-8da2-9c56e6c258af None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:45.560743 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 331/673] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:52:45 2022] PUT /identity/v3/projects/0ee65b9a8229495bbe97318f060d6342/users/7a59f547a20e48ee88e3e620ba1e87f1/roles/b53e713d4b4a460b9465321b090b0455 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:52:45.904531 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe982a-556d-4f6a-a36f-1f42ff4f06e7 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:45.905265 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe982a-556d-4f6a-a36f-1f42ff4f06e7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:45.905744 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe982a-556d-4f6a-a36f-1f42ff4f06e7 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:45.907517 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 343/674] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:45.913822 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c6fe8-673a-4d4c-81b8-1358765b554d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:45.914347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c6fe8-673a-4d4c-81b8-1358765b554d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:45.914828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c6fe8-673a-4d4c-81b8-1358765b554d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:45.933963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-093c6fe8-673a-4d4c-81b8-1358765b554d None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:45.935515 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-093c6fe8-673a-4d4c-81b8-1358765b554d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:46.008185 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 332/675] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:52:46.278895 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe982a-556d-4f6a-a36f-1f42ff4f06e7 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:46.323189 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:46.394752 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe982a-556d-4f6a-a36f-1f42ff4f06e7 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:46.428680 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282efaa-499f-47a8-9018-a50660e1e9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:46.429327 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282efaa-499f-47a8-9018-a50660e1e9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:46.429601 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282efaa-499f-47a8-9018-a50660e1e9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:46.429867 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282efaa-499f-47a8-9018-a50660e1e9c3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:46.430434 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6282efaa-499f-47a8-9018-a50660e1e9c3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:46.431778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6282efaa-499f-47a8-9018-a50660e1e9c3 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:46.440731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-6282efaa-499f-47a8-9018-a50660e1e9c3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 05:52:46.440731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:46.440731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:52:46.440731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:52:46.440731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:52:46.440731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:52:46.440731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:52:46.440731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:46.440731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 05:52:46.440731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:46.440731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:52:46.440731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:46.440731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:46.440731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:46.440731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:46.440731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:46.440731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:46.441712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:46.441712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:46.441712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:46.441712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:46.441712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:46.441712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:46.441712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:52:46.441712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:52:46.441712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:52:46.441712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:52:46.441712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:46.441712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:46.441712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:46.441712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:46.441712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:46.441712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:46.442925 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:46.442925 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:46.442925 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:46.442925 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:46.442925 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:46.442925 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:46.442925 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:46.442925 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:46.442925 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:46.442925 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:46.442925 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:52:46.442925 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:46.442925 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 05:52:46.442925 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:46.442925 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 344/676] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Oct 20 05:52:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:46.448089 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c6fe8-673a-4d4c-81b8-1358765b554d None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:46.493525 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:46.551908 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c6fe8-673a-4d4c-81b8-1358765b554d None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:46.586600 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9674f41-5b6c-4036-adbd-51aef33499f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:46.587204 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9674f41-5b6c-4036-adbd-51aef33499f8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:46.587477 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9674f41-5b6c-4036-adbd-51aef33499f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:46.587744 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9674f41-5b6c-4036-adbd-51aef33499f8 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:46.588363 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9674f41-5b6c-4036-adbd-51aef33499f8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:46.588659 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9674f41-5b6c-4036-adbd-51aef33499f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:46.590134 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9674f41-5b6c-4036-adbd-51aef33499f8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:46.595143 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 333/677] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Oct 20 05:52:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:46.601353 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282efaa-499f-47a8-9018-a50660e1e9c3 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:46.704561 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282efaa-499f-47a8-9018-a50660e1e9c3 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:46.709109 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:46.740406 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aeb607-2c5a-49d1-a820-ea5a953f74ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:46.741095 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aeb607-2c5a-49d1-a820-ea5a953f74ca None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:46.741377 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aeb607-2c5a-49d1-a820-ea5a953f74ca None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:46.741643 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aeb607-2c5a-49d1-a820-ea5a953f74ca None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:46.742260 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aeb607-2c5a-49d1-a820-ea5a953f74ca None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:46.742554 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aeb607-2c5a-49d1-a820-ea5a953f74ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:46.743914 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aeb607-2c5a-49d1-a820-ea5a953f74ca None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:46.749035 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 345/678] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Oct 20 05:52:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:46.755547 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9674f41-5b6c-4036-adbd-51aef33499f8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:46.856642 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9674f41-5b6c-4036-adbd-51aef33499f8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:46.858876 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:46.891821 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50fc99-f801-4550-8ac4-1dc7ccd3a4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:46.893070 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50fc99-f801-4550-8ac4-1dc7ccd3a4e7 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:46.893347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50fc99-f801-4550-8ac4-1dc7ccd3a4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:46.893612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50fc99-f801-4550-8ac4-1dc7ccd3a4e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:46.894294 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50fc99-f801-4550-8ac4-1dc7ccd3a4e7 None admin] RBAC: Adding query filter params (interface=public, service_id=2395e318c4ad4c4c9d0710a1c07f5245, region_id=RegionOne) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:46.894644 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50fc99-f801-4550-8ac4-1dc7ccd3a4e7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:46.895989 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50fc99-f801-4550-8ac4-1dc7ccd3a4e7 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:46.901373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 334/679] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Oct 20 05:52:46 2022] GET /identity/v3/endpoints?service_id=2395e318c4ad4c4c9d0710a1c07f5245&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:46.907134 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aeb607-2c5a-49d1-a820-ea5a953f74ca None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:47.009604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aeb607-2c5a-49d1-a820-ea5a953f74ca None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:47.044483 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5d2e0-199a-4f59-88e5-02218c613d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:47.045122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5d2e0-199a-4f59-88e5-02218c613d46 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:47.045397 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5d2e0-199a-4f59-88e5-02218c613d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:47.045663 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5d2e0-199a-4f59-88e5-02218c613d46 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:47.046316 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5d2e0-199a-4f59-88e5-02218c613d46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:47.047672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5d2e0-199a-4f59-88e5-02218c613d46 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:47.053948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 346/680] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Oct 20 05:52:46 2022] GET /identity/v3/services => generated 1497 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 05:52:47.601107 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa556d-cb8a-4a42-b65e-ac133806d8d1 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:47.601632 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa556d-cb8a-4a42-b65e-ac133806d8d1 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:47.601632 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa556d-cb8a-4a42-b65e-ac133806d8d1 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:47.603136 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 335/681] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:47.608308 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f543c-5712-49a8-a6c3-1523338cd8d0 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:47.608730 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f543c-5712-49a8-a6c3-1523338cd8d0 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:47.608920 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f543c-5712-49a8-a6c3-1523338cd8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:47.627871 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-6e4f543c-5712-49a8-a6c3-1523338cd8d0 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:47.629085 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6e4f543c-5712-49a8-a6c3-1523338cd8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:47.652724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:47.686221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd91ea-bbb2-4d55-85a1-880fe6df0645 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:47.686533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd91ea-bbb2-4d55-85a1-880fe6df0645 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:47.686761 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd91ea-bbb2-4d55-85a1-880fe6df0645 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:47.688293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 336/682] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:47.694631 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44c173-a728-4cdf-a896-e78efde4cf65 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:47.695463 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44c173-a728-4cdf-a896-e78efde4cf65 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:47.695934 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44c173-a728-4cdf-a896-e78efde4cf65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:47.705429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 347/683] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:52:47.718175 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-8e44c173-a728-4cdf-a896-e78efde4cf65 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:47.719742 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8e44c173-a728-4cdf-a896-e78efde4cf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:47.790320 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 337/684] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:52:47.813702 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac13f24-6f6a-4b00-8570-a7809a4e5c4e None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:47.814144 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac13f24-6f6a-4b00-8570-a7809a4e5c4e None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:47.814444 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac13f24-6f6a-4b00-8570-a7809a4e5c4e None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:47.816138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 348/685] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:47.821441 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce0198-3a3a-45f2-ba2f-532e0b219026 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:47.821973 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce0198-3a3a-45f2-ba2f-532e0b219026 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:47.822456 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce0198-3a3a-45f2-ba2f-532e0b219026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:47.841320 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-15ce0198-3a3a-45f2-ba2f-532e0b219026 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:47.842765 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-15ce0198-3a3a-45f2-ba2f-532e0b219026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:47.911122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 338/686] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:52:47.975622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac13f24-6f6a-4b00-8570-a7809a4e5c4e None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:48.074437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce0198-3a3a-45f2-ba2f-532e0b219026 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:48.082091 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac13f24-6f6a-4b00-8570-a7809a4e5c4e None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:48.104330 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:48.124939 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b9509-15c7-4fff-b1f8-bd393167ff38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:48.125570 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b9509-15c7-4fff-b1f8-bd393167ff38 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:48.125844 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b9509-15c7-4fff-b1f8-bd393167ff38 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:48.126110 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b9509-15c7-4fff-b1f8-bd393167ff38 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:48.127186 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b9509-15c7-4fff-b1f8-bd393167ff38 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:48.129076 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b9509-15c7-4fff-b1f8-bd393167ff38 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:48.144740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-7e9b9509-15c7-4fff-b1f8-bd393167ff38 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 05:52:48.144740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:48.144740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:52:48.144740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:52:48.144740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:52:48.144740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:52:48.144740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:52:48.144740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:48.144740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 05:52:48.144740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:48.144740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:52:48.144740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:48.144740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:48.144740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:48.144740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:48.144740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:48.144740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:48.145755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:48.145755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:48.145755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:48.145755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:48.145755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:48.145755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:48.145755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:52:48.145755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:52:48.145755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:52:48.145755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:52:48.145755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:48.145755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:48.145755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:48.145755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:48.145755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:48.145755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:48.146607 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:48.146607 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:48.146607 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:48.146607 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:48.146607 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:48.146607 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:48.146607 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:48.146607 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:48.146607 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:48.146607 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:48.146607 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:52:48.146607 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:48.146607 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 05:52:48.146607 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:48.149080 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 349/687] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Oct 20 05:52:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:48.159353 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b9509-15c7-4fff-b1f8-bd393167ff38 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:48.187660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:48.198088 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce0198-3a3a-45f2-ba2f-532e0b219026 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:48.235753 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbd70d-0237-450f-982b-54779a768fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:48.236964 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbd70d-0237-450f-982b-54779a768fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:48.237245 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbd70d-0237-450f-982b-54779a768fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:48.237521 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbd70d-0237-450f-982b-54779a768fe7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:48.238098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbbd70d-0237-450f-982b-54779a768fe7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:48.239462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbbd70d-0237-450f-982b-54779a768fe7 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:48.248336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-3cbbd70d-0237-450f-982b-54779a768fe7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 05:52:48.248336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:48.248336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:52:48.248336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:52:48.248336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:52:48.248336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:52:48.248336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:52:48.248336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:48.248336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 05:52:48.248336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:48.248336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:52:48.248336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:48.248336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:48.248336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:48.248336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:48.248336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:48.249348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:48.249348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:48.249348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:48.249348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:48.249348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:48.249348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:48.249348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:48.249348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:52:48.249348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:52:48.249348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:52:48.249348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:52:48.249348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:48.249348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:48.249348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:48.249348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:48.249348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:48.249348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:48.250235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:48.250235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:48.250235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:48.250235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:48.250235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:48.250235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:48.250235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:48.250235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:48.250235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:48.250235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:48.250235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:52:48.250235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:48.250235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 05:52:48.250235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:48.254467 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 339/688] 10.222.0.20 () {56 vars in 1132 bytes} [Thu Oct 20 05:52:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:52:48.255061 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbd70d-0237-450f-982b-54779a768fe7 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:48.275053 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b9509-15c7-4fff-b1f8-bd393167ff38 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:48.312618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdbeea6-29f6-4cbd-92e8-3ba72b5fd217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:48.313276 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbeea6-29f6-4cbd-92e8-3ba72b5fd217 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:48.313920 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbeea6-29f6-4cbd-92e8-3ba72b5fd217 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:48.313920 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbeea6-29f6-4cbd-92e8-3ba72b5fd217 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:48.314542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdbeea6-29f6-4cbd-92e8-3ba72b5fd217 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:48.314839 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdbeea6-29f6-4cbd-92e8-3ba72b5fd217 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:48.316206 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdbeea6-29f6-4cbd-92e8-3ba72b5fd217 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:48.321079 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 350/689] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Oct 20 05:52:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:48.326166 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdbeea6-29f6-4cbd-92e8-3ba72b5fd217 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:48.358108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:48.368120 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbd70d-0237-450f-982b-54779a768fe7 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:48.403787 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ff951-1e4a-4f3c-b2a9-0ea6cf9719c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:48.405233 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ff951-1e4a-4f3c-b2a9-0ea6cf9719c2 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:48.405518 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ff951-1e4a-4f3c-b2a9-0ea6cf9719c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:48.405794 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ff951-1e4a-4f3c-b2a9-0ea6cf9719c2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:48.418589 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ff951-1e4a-4f3c-b2a9-0ea6cf9719c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:52:48.420335 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:48.424260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-ad6ff951-1e4a-4f3c-b2a9-0ea6cf9719c2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 20 05:52:48.424260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:48.424260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:48.424260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:48.424260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:48.424260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:48.424260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:48.424260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:48.424260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:48.424260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:48.424260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:48.424260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:48.424260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:52:48.424260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:52:48.424260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:52:48.424260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:52:48.425248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:52:48.425248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:52:48.425248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:52:48.425248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:52:48.425248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:48.425248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:48.425248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:48.425248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:48.425248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:48.425248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:48.425248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:48.425248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:48.425248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:48.425248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:48.425248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:48.425248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:48.426108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:48.426108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:48.426108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:48.426108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:48.426108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:52:48.426108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:52:48.426108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:52:48.426108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:52:48.426108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:52:48.426108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:52:48.426108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 20 05:52:48.426108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:48.426108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 340/690] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Oct 20 05:52:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:48.426807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ff951-1e4a-4f3c-b2a9-0ea6cf9719c2 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:48.444317 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdbeea6-29f6-4cbd-92e8-3ba72b5fd217 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:48.485751 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b55b8c-9bfc-4105-b3e0-1c661872cc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:48.486809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b55b8c-9bfc-4105-b3e0-1c661872cc33 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:48.487080 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b55b8c-9bfc-4105-b3e0-1c661872cc33 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:48.487400 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b55b8c-9bfc-4105-b3e0-1c661872cc33 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:48.488006 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b55b8c-9bfc-4105-b3e0-1c661872cc33 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:48.488300 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b55b8c-9bfc-4105-b3e0-1c661872cc33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:48.490539 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b55b8c-9bfc-4105-b3e0-1c661872cc33 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:48.500389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 351/691] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Oct 20 05:52:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:48.500389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b55b8c-9bfc-4105-b3e0-1c661872cc33 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:48.531551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ff951-1e4a-4f3c-b2a9-0ea6cf9719c2 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:48.569178 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c65f9-036c-4e35-a602-ba6609ebb94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:48.569802 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c65f9-036c-4e35-a602-ba6609ebb94d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:48.570084 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c65f9-036c-4e35-a602-ba6609ebb94d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:48.570354 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c65f9-036c-4e35-a602-ba6609ebb94d None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:48.570980 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c65f9-036c-4e35-a602-ba6609ebb94d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:48.571278 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c65f9-036c-4e35-a602-ba6609ebb94d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:48.572625 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c65f9-036c-4e35-a602-ba6609ebb94d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:48.582440 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 341/692] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Oct 20 05:52:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:48.582440 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c65f9-036c-4e35-a602-ba6609ebb94d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:48.600839 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:48.609208 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b55b8c-9bfc-4105-b3e0-1c661872cc33 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:48.646539 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e199e-d30f-403d-a570-e5a08f970ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:48.647144 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e199e-d30f-403d-a570-e5a08f970ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:48.647421 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e199e-d30f-403d-a570-e5a08f970ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:48.647694 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e199e-d30f-403d-a570-e5a08f970ec2 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:48.648455 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e199e-d30f-403d-a570-e5a08f970ec2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:48.648898 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e199e-d30f-403d-a570-e5a08f970ec2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:48.650257 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e199e-d30f-403d-a570-e5a08f970ec2 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:48.663556 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 352/693] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Oct 20 05:52:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:48.663556 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e199e-d30f-403d-a570-e5a08f970ec2 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:48.679703 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:48.690965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c65f9-036c-4e35-a602-ba6609ebb94d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:48.726023 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31086e67-26ea-4684-9c9a-c05a127a924f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:48.726608 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31086e67-26ea-4684-9c9a-c05a127a924f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:48.726894 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31086e67-26ea-4684-9c9a-c05a127a924f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:48.727171 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31086e67-26ea-4684-9c9a-c05a127a924f None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:48.727786 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31086e67-26ea-4684-9c9a-c05a127a924f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:48.728093 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31086e67-26ea-4684-9c9a-c05a127a924f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:48.729486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31086e67-26ea-4684-9c9a-c05a127a924f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:48.734155 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 342/694] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Oct 20 05:52:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:48.734912 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31086e67-26ea-4684-9c9a-c05a127a924f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:48.781259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e199e-d30f-403d-a570-e5a08f970ec2 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:48.815597 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:48.819542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db5abf-f838-4106-a222-94a7825410b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:48.820355 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db5abf-f838-4106-a222-94a7825410b5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:48.820832 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db5abf-f838-4106-a222-94a7825410b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:48.821297 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db5abf-f838-4106-a222-94a7825410b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:48.822158 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db5abf-f838-4106-a222-94a7825410b5 None admin] RBAC: Adding query filter params (interface=public, service_id=bb17fc02f2474731b24de57187de0235, region_id=RegionOne) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:48.822633 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db5abf-f838-4106-a222-94a7825410b5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:48.824151 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db5abf-f838-4106-a222-94a7825410b5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:48.829544 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 353/695] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Oct 20 05:52:48 2022] GET /identity/v3/endpoints?service_id=bb17fc02f2474731b24de57187de0235&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:48.830635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db5abf-f838-4106-a222-94a7825410b5 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:48.839987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31086e67-26ea-4684-9c9a-c05a127a924f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:48.874088 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37b3cf-6cbc-400d-983d-e3b742ca9623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:48.874722 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37b3cf-6cbc-400d-983d-e3b742ca9623 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:48.875001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37b3cf-6cbc-400d-983d-e3b742ca9623 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:48.875269 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37b3cf-6cbc-400d-983d-e3b742ca9623 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:48.886598 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a37b3cf-6cbc-400d-983d-e3b742ca9623 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:48.888037 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a37b3cf-6cbc-400d-983d-e3b742ca9623 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:48.902300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-6a37b3cf-6cbc-400d-983d-e3b742ca9623 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:48.902300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:48.902300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:48.902300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:48.902300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:48.902300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:48.902300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:48.902300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:48.902300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:48.902300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:48.902300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:48.902300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:48.902300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:48.902300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:48.902300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:48.902300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:48.903276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:48.903276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:48.903276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:48.903276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:48.903276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:48.903276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:48.903276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:48.903276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:48.903276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:48.903276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:48.903276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:48.903276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:48.903276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:48.903276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:48.903276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:48.903276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:48.903276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:48.904190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:48.904190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:48.904190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:48.904190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 343/696] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:48.904190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37b3cf-6cbc-400d-983d-e3b742ca9623 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:48.919995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:48.945552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db5abf-f838-4106-a222-94a7825410b5 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:48.983311 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefc696-ba27-4f49-9464-50e4eb0be746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:48.983837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefc696-ba27-4f49-9464-50e4eb0be746 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:48.984108 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefc696-ba27-4f49-9464-50e4eb0be746 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:48.984374 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefc696-ba27-4f49-9464-50e4eb0be746 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:48.985074 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefc696-ba27-4f49-9464-50e4eb0be746 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:48.986972 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefc696-ba27-4f49-9464-50e4eb0be746 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:49.007708 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 354/697] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 05:52:48 2022] POST /identity/v3/endpoints => generated 347 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:52:49.008672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefc696-ba27-4f49-9464-50e4eb0be746 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:49.036750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37b3cf-6cbc-400d-983d-e3b742ca9623 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:49.084870 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:49.101214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db15d4-c2a4-4198-b7b4-d44c94edc055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:49.102220 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db15d4-c2a4-4198-b7b4-d44c94edc055 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:49.103451 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db15d4-c2a4-4198-b7b4-d44c94edc055 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:49.103931 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db15d4-c2a4-4198-b7b4-d44c94edc055 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:49.104776 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db15d4-c2a4-4198-b7b4-d44c94edc055 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:49.107548 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db15d4-c2a4-4198-b7b4-d44c94edc055 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:49.117197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 344/698] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Oct 20 05:52:48 2022] GET /identity/v3/services => generated 1497 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 05:52:49.142764 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefc696-ba27-4f49-9464-50e4eb0be746 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:49.184272 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:49.192046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ae8e6-19ff-43e3-a12f-e17e694a2ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:49.193539 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ae8e6-19ff-43e3-a12f-e17e694a2ac9 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:49.194459 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ae8e6-19ff-43e3-a12f-e17e694a2ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:49.194930 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ae8e6-19ff-43e3-a12f-e17e694a2ac9 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:49.195880 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ae8e6-19ff-43e3-a12f-e17e694a2ac9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:49.196377 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ae8e6-19ff-43e3-a12f-e17e694a2ac9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:49.198024 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ae8e6-19ff-43e3-a12f-e17e694a2ac9 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:49.215164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 355/699] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:49.221740 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db15d4-c2a4-4198-b7b4-d44c94edc055 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:49.295618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953f791-935b-407d-9160-05a1f628847e None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:49.295795 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953f791-935b-407d-9160-05a1f628847e None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:49.296070 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953f791-935b-407d-9160-05a1f628847e None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:49.297688 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 356/700] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:52:49.303566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec62ab-e179-4d8d-b6f6-9423a6e50370 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:49.303844 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec62ab-e179-4d8d-b6f6-9423a6e50370 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:49.304624 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec62ab-e179-4d8d-b6f6-9423a6e50370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:49.324050 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-44ec62ab-e179-4d8d-b6f6-9423a6e50370 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:49.325247 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-44ec62ab-e179-4d8d-b6f6-9423a6e50370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:49.347672 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db15d4-c2a4-4198-b7b4-d44c94edc055 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:49.385236 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd5aea-9adb-4050-b1ca-1ef4a770f365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:49.385883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd5aea-9adb-4050-b1ca-1ef4a770f365 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:49.386163 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd5aea-9adb-4050-b1ca-1ef4a770f365 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:49.386437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd5aea-9adb-4050-b1ca-1ef4a770f365 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:49.394801 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd5aea-9adb-4050-b1ca-1ef4a770f365 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:49.396247 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd5aea-9adb-4050-b1ca-1ef4a770f365 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:49.408428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-43dd5aea-9adb-4050-b1ca-1ef4a770f365 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 20 05:52:49.408428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:49.408428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:49.408428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:49.408428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:49.408428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:49.408428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:49.408428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:49.408428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:49.408428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:49.408428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:49.408428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:49.408428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:52:49.408428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:52:49.408428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:52:49.408428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:52:49.409486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:49.409486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:49.409486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:52:49.409486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:49.409486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:52:49.409486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:49.409486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:49.409486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:49.409486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:49.409486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:49.409486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:49.409486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:49.409486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:49.409486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:49.409486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:49.409486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:49.409486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:49.409486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:49.410437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:49.410437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:49.410437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:52:49.410437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:52:49.410437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:52:49.410437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:52:49.410437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:52:49.410437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:52:49.410437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 20 05:52:49.410437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:49.410437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 345/701] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Oct 20 05:52:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:49.415389 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd5aea-9adb-4050-b1ca-1ef4a770f365 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:49.416616 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 357/702] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 05:52:49.470858 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:49.522021 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd5aea-9adb-4050-b1ca-1ef4a770f365 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:49.562091 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029b1fa-c8fc-41ea-a15e-18fc2c8dd5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:49.562677 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029b1fa-c8fc-41ea-a15e-18fc2c8dd5e4 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:49.562958 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029b1fa-c8fc-41ea-a15e-18fc2c8dd5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:49.563232 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029b1fa-c8fc-41ea-a15e-18fc2c8dd5e4 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:49.564481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c029b1fa-c8fc-41ea-a15e-18fc2c8dd5e4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:49.564828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c029b1fa-c8fc-41ea-a15e-18fc2c8dd5e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:49.566266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c029b1fa-c8fc-41ea-a15e-18fc2c8dd5e4 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:49.584273 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 346/703] 10.222.0.20 () {56 vars in 1149 bytes} [Thu Oct 20 05:52:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:49.590200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec62ab-e179-4d8d-b6f6-9423a6e50370 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:49.691758 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec62ab-e179-4d8d-b6f6-9423a6e50370 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:49.692405 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029b1fa-c8fc-41ea-a15e-18fc2c8dd5e4 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:49.719620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:49.737031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e9c995-b291-44c2-9ea4-3cd83089e258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:49.737650 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9c995-b291-44c2-9ea4-3cd83089e258 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:49.737925 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9c995-b291-44c2-9ea4-3cd83089e258 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:49.738193 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9c995-b291-44c2-9ea4-3cd83089e258 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:49.806679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029b1fa-c8fc-41ea-a15e-18fc2c8dd5e4 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:49.819065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:49.827612 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e9c995-b291-44c2-9ea4-3cd83089e258 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:49.830483 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e9c995-b291-44c2-9ea4-3cd83089e258 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:49.844289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc1095-81ca-4a3a-a290-78974fe5c058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:49.845081 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc1095-81ca-4a3a-a290-78974fe5c058 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:49.846110 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc1095-81ca-4a3a-a290-78974fe5c058 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:49.846474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc1095-81ca-4a3a-a290-78974fe5c058 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:49.847148 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc1095-81ca-4a3a-a290-78974fe5c058 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:49.848649 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc1095-81ca-4a3a-a290-78974fe5c058 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:49.858057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-8ddc1095-81ca-4a3a-a290-78974fe5c058 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 05:52:49.858057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:49.858057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:52:49.858057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:52:49.858057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:52:49.858057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:52:49.858057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:52:49.858057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:49.858057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 05:52:49.858057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:49.858057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:52:49.858057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:49.858057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:49.858057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:49.858057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:49.858057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:49.858057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:49.859948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:49.859948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:49.859948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:49.859948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:49.859948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:49.859948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:49.859948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:52:49.859948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:52:49.859948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:52:49.859948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:52:49.859948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:49.859948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:49.859948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:49.859948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:49.859948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:49.859948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:49.861506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:49.861506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:49.861506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:49.861506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:49.861506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:49.861506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:49.861506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:49.861506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:49.861506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:49.861506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:49.861506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:52:49.861506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:49.861506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 05:52:49.861506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:49.861506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 347/704] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Oct 20 05:52:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:52:49.873701 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc1095-81ca-4a3a-a290-78974fe5c058 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:49.945463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 358/705] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:52:49 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 05:52:49.950597 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e9c995-b291-44c2-9ea4-3cd83089e258 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:49.977226 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:50.016466 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc1095-81ca-4a3a-a290-78974fe5c058 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:50.059379 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e9c995-b291-44c2-9ea4-3cd83089e258 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:50.065442 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea3b38-0d36-46e0-af3a-255af590549f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:50.066289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea3b38-0d36-46e0-af3a-255af590549f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:50.067317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea3b38-0d36-46e0-af3a-255af590549f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:50.067788 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea3b38-0d36-46e0-af3a-255af590549f None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:50.068601 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea3b38-0d36-46e0-af3a-255af590549f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:50.069644 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea3b38-0d36-46e0-af3a-255af590549f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:50.071782 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea3b38-0d36-46e0-af3a-255af590549f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:50.086475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 348/706] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Oct 20 05:52:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:50.087340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea3b38-0d36-46e0-af3a-255af590549f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:50.106776 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66950e19-ad04-4475-942b-a4fb1f353a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:50.108117 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66950e19-ad04-4475-942b-a4fb1f353a48 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:50.108614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66950e19-ad04-4475-942b-a4fb1f353a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:50.109186 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66950e19-ad04-4475-942b-a4fb1f353a48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:50.125545 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66950e19-ad04-4475-942b-a4fb1f353a48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:50.129432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66950e19-ad04-4475-942b-a4fb1f353a48 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:50.136172 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:50.147460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-66950e19-ad04-4475-942b-a4fb1f353a48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:50.147460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:50.147460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:50.147460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:50.147460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:50.147460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:50.147460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:50.147460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:50.147460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:50.147460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:50.147460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:50.147460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:50.147460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:52:50.147460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:52:50.147460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:52:50.147460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:52:50.148429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:50.148429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:50.148429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:50.148429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:50.148429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:50.148429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:50.148429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:50.148429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:50.148429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:50.148429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:50.148429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:50.148429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:50.148429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:50.148429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:50.148429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:50.148429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:50.148429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:52:50.149364 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:52:50.149364 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:52:50.149364 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:50.153144 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 359/707] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:52:50.154466 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66950e19-ad04-4475-942b-a4fb1f353a48 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:50.217022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea3b38-0d36-46e0-af3a-255af590549f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:50.238037 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:50.263489 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145dd900-db0d-4514-927f-54163bedf427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:50.264327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145dd900-db0d-4514-927f-54163bedf427 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:50.264602 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145dd900-db0d-4514-927f-54163bedf427 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:50.264896 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145dd900-db0d-4514-927f-54163bedf427 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:50.265518 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145dd900-db0d-4514-927f-54163bedf427 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:50.265814 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145dd900-db0d-4514-927f-54163bedf427 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:50.267696 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145dd900-db0d-4514-927f-54163bedf427 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:50.270629 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66950e19-ad04-4475-942b-a4fb1f353a48 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:50.278447 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 349/708] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Oct 20 05:52:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:50.284443 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145dd900-db0d-4514-927f-54163bedf427 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:50.319851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13ac90-6bf7-4787-afca-9cacf47995c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:50.320534 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13ac90-6bf7-4787-afca-9cacf47995c3 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:50.320904 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13ac90-6bf7-4787-afca-9cacf47995c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:50.321169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13ac90-6bf7-4787-afca-9cacf47995c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:50.321864 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb13ac90-6bf7-4787-afca-9cacf47995c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:50.322148 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb13ac90-6bf7-4787-afca-9cacf47995c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:50.323640 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb13ac90-6bf7-4787-afca-9cacf47995c3 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:50.342009 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 360/709] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:50.343243 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13ac90-6bf7-4787-afca-9cacf47995c3 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:50.413352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145dd900-db0d-4514-927f-54163bedf427 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:50.417291 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:50.449466 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13ac90-6bf7-4787-afca-9cacf47995c3 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:50.450837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc0518-0617-4b6c-8ee4-d18f3305b9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:50.451681 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc0518-0617-4b6c-8ee4-d18f3305b9d6 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:50.452186 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc0518-0617-4b6c-8ee4-d18f3305b9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:50.452659 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc0518-0617-4b6c-8ee4-d18f3305b9d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:50.453639 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc0518-0617-4b6c-8ee4-d18f3305b9d6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:50.455582 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc0518-0617-4b6c-8ee4-d18f3305b9d6 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:50.478414 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 350/710] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 05:52:50 2022] POST /identity/v3/endpoints => generated 349 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 05:52:50.500688 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad18de3c-fffd-4a87-9f52-39a82aed119b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:50.502304 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18de3c-fffd-4a87-9f52-39a82aed119b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:50.502793 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18de3c-fffd-4a87-9f52-39a82aed119b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:50.503263 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18de3c-fffd-4a87-9f52-39a82aed119b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:50.511751 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:50.512334 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad18de3c-fffd-4a87-9f52-39a82aed119b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:50.514186 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad18de3c-fffd-4a87-9f52-39a82aed119b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:50.527761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-ad18de3c-fffd-4a87-9f52-39a82aed119b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:50.527761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:50.527761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:50.527761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:50.527761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:50.527761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:50.527761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:50.527761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:50.527761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:50.527761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:50.527761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:50.527761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:50.527761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:52:50.527761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:52:50.527761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:52:50.528763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:52:50.528763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:50.528763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:50.528763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:50.528763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:50.528763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:50.528763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:50.528763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:50.528763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:50.528763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:50.528763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:50.528763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:50.528763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:50.528763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:50.528763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:50.528763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:50.528763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:50.529699 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:52:50.529699 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:52:50.529699 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:52:50.529699 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:52:50.529699 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:52:50.529699 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:52:50.529699 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:50.529699 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:50.532216 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 361/711] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:50.538378 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc0518-0617-4b6c-8ee4-d18f3305b9d6 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:50.652688 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc0518-0617-4b6c-8ee4-d18f3305b9d6 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:50.688595 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414041ef-6760-421a-b1e3-e760793ab437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:50.689336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414041ef-6760-421a-b1e3-e760793ab437 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:50.689621 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414041ef-6760-421a-b1e3-e760793ab437 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:50.689894 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414041ef-6760-421a-b1e3-e760793ab437 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:50.690578 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414041ef-6760-421a-b1e3-e760793ab437 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:50.690883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414041ef-6760-421a-b1e3-e760793ab437 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:50.692302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414041ef-6760-421a-b1e3-e760793ab437 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:50.705200 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 351/712] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Oct 20 05:52:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:50.712051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad18de3c-fffd-4a87-9f52-39a82aed119b None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:50.822422 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad18de3c-fffd-4a87-9f52-39a82aed119b None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:50.858592 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31064e7-123d-4ca6-8ba9-f90e29281741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:50.859203 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31064e7-123d-4ca6-8ba9-f90e29281741 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:50.859542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31064e7-123d-4ca6-8ba9-f90e29281741 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:50.859823 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31064e7-123d-4ca6-8ba9-f90e29281741 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:50.860548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31064e7-123d-4ca6-8ba9-f90e29281741 None admin] RBAC: Adding query filter params (role.id=b53e713d4b4a460b9465321b090b0455, scope.project.id=0ee65b9a8229495bbe97318f060d6342, user.id=7a59f547a20e48ee88e3e620ba1e87f1) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:50.861410 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31064e7-123d-4ca6-8ba9-f90e29281741 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:50.862844 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31064e7-123d-4ca6-8ba9-f90e29281741 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:50.869530 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 362/713] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:52:50 2022] GET /identity/v3/role_assignments?user.id=7a59f547a20e48ee88e3e620ba1e87f1&scope.project.id=0ee65b9a8229495bbe97318f060d6342&role.id=b53e713d4b4a460b9465321b090b0455 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:51.350700 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f7173-c15d-4b29-880e-0480ee93f296 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:51.350983 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f7173-c15d-4b29-880e-0480ee93f296 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:51.351263 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f7173-c15d-4b29-880e-0480ee93f296 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:51.352975 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 352/714] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:51.358909 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d12aa-490a-449f-ae02-4cc011053fa4 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:51.359196 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d12aa-490a-449f-ae02-4cc011053fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:51.359470 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d12aa-490a-449f-ae02-4cc011053fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:51.380085 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-119d12aa-490a-449f-ae02-4cc011053fa4 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:51.381316 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-119d12aa-490a-449f-ae02-4cc011053fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:51.437999 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:51.457351 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 363/715] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:52:51.463131 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062ced0-4117-43b5-bce5-8184ebc1a6b3 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:51.463417 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062ced0-4117-43b5-bce5-8184ebc1a6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:51.463695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062ced0-4117-43b5-bce5-8184ebc1a6b3 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:51.465438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 353/716] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:51.471520 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db25a3-5698-4045-8e87-ce79b56c969d None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:51.472155 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db25a3-5698-4045-8e87-ce79b56c969d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:51.472626 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db25a3-5698-4045-8e87-ce79b56c969d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:51.491826 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-b1db25a3-5698-4045-8e87-ce79b56c969d None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:51.493333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b1db25a3-5698-4045-8e87-ce79b56c969d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:51.564667 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 364/717] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:52:51.734416 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062ced0-4117-43b5-bce5-8184ebc1a6b3 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:51.754513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:51.846804 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062ced0-4117-43b5-bce5-8184ebc1a6b3 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:51.860346 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db25a3-5698-4045-8e87-ce79b56c969d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:51.884020 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27516da-d7f4-4fd1-8076-f7b0ba4b2c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:51.884649 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27516da-d7f4-4fd1-8076-f7b0ba4b2c91 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:51.885188 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27516da-d7f4-4fd1-8076-f7b0ba4b2c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:51.885465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27516da-d7f4-4fd1-8076-f7b0ba4b2c91 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:51.886033 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27516da-d7f4-4fd1-8076-f7b0ba4b2c91 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:51.887407 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27516da-d7f4-4fd1-8076-f7b0ba4b2c91 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:51.896963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-e27516da-d7f4-4fd1-8076-f7b0ba4b2c91 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 05:52:51.896963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:51.896963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:52:51.896963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:52:51.896963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:52:51.896963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:52:51.896963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:52:51.896963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:51.896963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 05:52:51.896963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:51.896963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:52:51.896963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:51.896963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:51.896963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:51.896963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:51.896963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:51.896963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:51.897967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:51.897967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:51.897967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:51.897967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:51.897967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:51.897967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:51.897967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:52:51.897967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:52:51.897967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:52:51.897967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:52:51.897967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:51.897967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:51.897967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:51.897967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:51.897967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:51.897967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:51.898838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:51.898838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:51.898838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:51.898838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:51.898838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:51.898838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:51.898838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:51.898838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:51.898838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:51.898838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:51.898838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:52:51.898838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:51.898838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 05:52:51.898838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:51.898838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 354/718] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:51.909649 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27516da-d7f4-4fd1-8076-f7b0ba4b2c91 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:51.985570 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db25a3-5698-4045-8e87-ce79b56c969d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:51.999558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:52.004434 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:52.026084 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27516da-d7f4-4fd1-8076-f7b0ba4b2c91 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:52.031121 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd1462-1210-450e-8fd5-e505dd5c64db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:52.031853 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd1462-1210-450e-8fd5-e505dd5c64db None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:52.032204 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd1462-1210-450e-8fd5-e505dd5c64db None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:52.032545 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd1462-1210-450e-8fd5-e505dd5c64db None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:52.033256 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd1462-1210-450e-8fd5-e505dd5c64db None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:52.034703 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd1462-1210-450e-8fd5-e505dd5c64db None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:52.044463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-3abd1462-1210-450e-8fd5-e505dd5c64db None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 05:52:52.044463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:52.044463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:52:52.044463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:52:52.044463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:52:52.044463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:52:52.044463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:52:52.044463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:52.044463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 05:52:52.044463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:52.044463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:52:52.044463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:52.044463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:52.044463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:52.044463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:52.044463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:52.044463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:52.045491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:52.045491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:52.045491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:52.045491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:52.045491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:52.045491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:52.045491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:52:52.045491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:52:52.045491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:52:52.045491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:52:52.045491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:52.045491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:52.045491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:52.045491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:52.045491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:52.045491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:52.046332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:52.046332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:52.046332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:52.046332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:52.046332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:52.046332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:52.046332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:52.046332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:52.046332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:52.046332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:52.046332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:52:52.046332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:52.046332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 05:52:52.046332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:52.051138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 365/719] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Oct 20 05:52:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:52.052071 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd1462-1210-450e-8fd5-e505dd5c64db None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:52.070858 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048465d-da1a-4aed-a184-1fc9525e8e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:52.071832 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048465d-da1a-4aed-a184-1fc9525e8e97 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:52.072282 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048465d-da1a-4aed-a184-1fc9525e8e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:52.072889 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048465d-da1a-4aed-a184-1fc9525e8e97 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:52.073966 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4048465d-da1a-4aed-a184-1fc9525e8e97 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:52.074439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4048465d-da1a-4aed-a184-1fc9525e8e97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:52.075976 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4048465d-da1a-4aed-a184-1fc9525e8e97 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:52.082603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 355/720] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Oct 20 05:52:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:52.088810 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048465d-da1a-4aed-a184-1fc9525e8e97 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:52.167995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd1462-1210-450e-8fd5-e505dd5c64db None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:52.211404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e2f1d-a4a4-48a0-8487-a96550e5b1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:52.212020 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e2f1d-a4a4-48a0-8487-a96550e5b1f2 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:52.212295 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e2f1d-a4a4-48a0-8487-a96550e5b1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:52.212566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e2f1d-a4a4-48a0-8487-a96550e5b1f2 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:52.213217 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e2f1d-a4a4-48a0-8487-a96550e5b1f2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:52.213517 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e2f1d-a4a4-48a0-8487-a96550e5b1f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:52.214876 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e2f1d-a4a4-48a0-8487-a96550e5b1f2 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:52.226466 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 366/721] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Oct 20 05:52:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:52.228037 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e2f1d-a4a4-48a0-8487-a96550e5b1f2 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:52.232397 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048465d-da1a-4aed-a184-1fc9525e8e97 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:52.269680 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f187b1-992d-4398-ba4c-aa39ad2986af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:52.270459 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f187b1-992d-4398-ba4c-aa39ad2986af None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:52.270879 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f187b1-992d-4398-ba4c-aa39ad2986af None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:52.271457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f187b1-992d-4398-ba4c-aa39ad2986af None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:52.272009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f187b1-992d-4398-ba4c-aa39ad2986af None admin] RBAC: Adding query filter params (type=compute) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:52.272409 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f187b1-992d-4398-ba4c-aa39ad2986af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:52.273958 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f187b1-992d-4398-ba4c-aa39ad2986af None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:52.285695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 356/722] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Oct 20 05:52:52 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:52.287808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:52.361619 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e2f1d-a4a4-48a0-8487-a96550e5b1f2 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:52.400584 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c6610-ffbe-4a5c-8998-d0f158943186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:52.401521 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c6610-ffbe-4a5c-8998-d0f158943186 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:52.401943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c6610-ffbe-4a5c-8998-d0f158943186 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:52.402353 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c6610-ffbe-4a5c-8998-d0f158943186 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:52.403131 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c6610-ffbe-4a5c-8998-d0f158943186 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:52.403572 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c6610-ffbe-4a5c-8998-d0f158943186 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:52.405152 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c6610-ffbe-4a5c-8998-d0f158943186 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:52.416414 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 367/723] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Oct 20 05:52:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:52.418956 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f187b1-992d-4398-ba4c-aa39ad2986af None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:52.435105 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:52.527485 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f187b1-992d-4398-ba4c-aa39ad2986af None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:52.561469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8d825-4690-4648-84e1-3150e18e0fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:52.562377 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8d825-4690-4648-84e1-3150e18e0fee None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:52.562830 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8d825-4690-4648-84e1-3150e18e0fee None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:52.563273 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8d825-4690-4648-84e1-3150e18e0fee None admin] PATH_INFO: `/v3/endpoints` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:52.564621 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8d825-4690-4648-84e1-3150e18e0fee None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:52.566196 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8d825-4690-4648-84e1-3150e18e0fee None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:52.592803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 357/724] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 05:52:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:52:52.800752 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f090e-6b4c-4e0e-a3d6-cf0594092626 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:52.801088 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f090e-6b4c-4e0e-a3d6-cf0594092626 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:52.801278 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f090e-6b4c-4e0e-a3d6-cf0594092626 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:52.802853 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 368/725] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:52.808221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e2657-33ee-48db-8500-17ef138d6594 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:52.808671 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e2657-33ee-48db-8500-17ef138d6594 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:52.808846 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e2657-33ee-48db-8500-17ef138d6594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:52.827910 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-be7e2657-33ee-48db-8500-17ef138d6594 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:52.829145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-be7e2657-33ee-48db-8500-17ef138d6594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:52.897168 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:52.912416 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 358/726] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:52:53.101996 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef2596-21d2-4a9d-a311-c59929bae81a None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:53.102483 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef2596-21d2-4a9d-a311-c59929bae81a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:53.102483 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef2596-21d2-4a9d-a311-c59929bae81a None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:53.104008 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 369/727] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:53.109341 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece07a64-449f-403a-af30-914f04e7a5a8 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:53.109624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece07a64-449f-403a-af30-914f04e7a5a8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:53.109895 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece07a64-449f-403a-af30-914f04e7a5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:53.128103 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-ece07a64-449f-403a-af30-914f04e7a5a8 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:53.129320 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ece07a64-449f-403a-af30-914f04e7a5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:53.198996 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ef2596-21d2-4a9d-a311-c59929bae81a None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:53.200175 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 359/728] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:52:53.314662 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ef2596-21d2-4a9d-a311-c59929bae81a None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:53.322055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:53.350935 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350609e-fc02-48c0-ab22-dc1341b0e6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:53.351628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350609e-fc02-48c0-ab22-dc1341b0e6aa None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:53.351912 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350609e-fc02-48c0-ab22-dc1341b0e6aa None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:53.352244 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350609e-fc02-48c0-ab22-dc1341b0e6aa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:53.352801 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350609e-fc02-48c0-ab22-dc1341b0e6aa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:53.354275 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350609e-fc02-48c0-ab22-dc1341b0e6aa None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:53.363698 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-5350609e-fc02-48c0-ab22-dc1341b0e6aa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 05:52:53.363698 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:53.363698 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:52:53.363698 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:52:53.363698 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:52:53.363698 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:52:53.363698 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:52:53.363698 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:53.363698 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 05:52:53.363698 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:53.363698 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:52:53.363698 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:53.363698 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:53.363698 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:53.363698 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:53.363698 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:53.363698 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:53.364737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:53.364737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:53.364737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:53.364737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:53.364737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:53.364737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:53.364737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:52:53.364737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:52:53.364737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:52:53.364737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:52:53.364737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:53.364737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:53.364737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:53.364737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:53.364737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:53.364737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:53.365577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:53.365577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:53.365577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:53.365577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:53.365577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:53.365577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:53.365577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:53.365577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:53.365577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:53.365577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:53.365577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:52:53.365577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:53.365577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 05:52:53.365577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:53.365577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 370/729] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Oct 20 05:52:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:52:53.371138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece07a64-449f-403a-af30-914f04e7a5a8 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:53.479795 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece07a64-449f-403a-af30-914f04e7a5a8 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:53.505044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:53.516204 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305c675-9a1c-447f-85e6-59bfd22b7db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:53.516890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305c675-9a1c-447f-85e6-59bfd22b7db9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:53.517196 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305c675-9a1c-447f-85e6-59bfd22b7db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:53.517472 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305c675-9a1c-447f-85e6-59bfd22b7db9 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:53.518141 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305c675-9a1c-447f-85e6-59bfd22b7db9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:53.518672 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305c675-9a1c-447f-85e6-59bfd22b7db9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:53.519584 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350609e-fc02-48c0-ab22-dc1341b0e6aa None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:53.522281 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305c675-9a1c-447f-85e6-59bfd22b7db9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:53.528160 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 360/730] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Oct 20 05:52:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:53.534967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305c675-9a1c-447f-85e6-59bfd22b7db9 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:53.651769 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305c675-9a1c-447f-85e6-59bfd22b7db9 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:53.693795 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1a1aa-0c5b-4247-9be3-3b12bf21c1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:53.694802 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1a1aa-0c5b-4247-9be3-3b12bf21c1db None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:53.695144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1a1aa-0c5b-4247-9be3-3b12bf21c1db None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:53.695427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1a1aa-0c5b-4247-9be3-3b12bf21c1db None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:53.696053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1a1aa-0c5b-4247-9be3-3b12bf21c1db None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:53.696358 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1a1aa-0c5b-4247-9be3-3b12bf21c1db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:53.698065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1a1aa-0c5b-4247-9be3-3b12bf21c1db None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:53.709145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 361/731] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Oct 20 05:52:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:53.710629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1a1aa-0c5b-4247-9be3-3b12bf21c1db None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:53.739642 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350609e-fc02-48c0-ab22-dc1341b0e6aa None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:53.805186 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:53.825863 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a21f7b-ed01-4a3f-a268-f328d40e81d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:53.826468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1a1aa-0c5b-4247-9be3-3b12bf21c1db None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:53.827401 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a21f7b-ed01-4a3f-a268-f328d40e81d8 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:53.827898 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a21f7b-ed01-4a3f-a268-f328d40e81d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:53.828406 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a21f7b-ed01-4a3f-a268-f328d40e81d8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:53.829256 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a21f7b-ed01-4a3f-a268-f328d40e81d8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:53.831827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a21f7b-ed01-4a3f-a268-f328d40e81d8 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:53.840968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-79a21f7b-ed01-4a3f-a268-f328d40e81d8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 05:52:53.840968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:53.840968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:52:53.840968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:52:53.840968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:52:53.840968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:52:53.840968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:52:53.840968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:53.840968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 05:52:53.840968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:53.840968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:52:53.840968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:53.840968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:53.840968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:53.840968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:53.840968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:53.840968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:53.841948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:53.841948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:53.841948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:53.841948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:53.841948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:53.841948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:53.841948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:52:53.841948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:52:53.841948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:52:53.841948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:52:53.841948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:53.841948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:53.841948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:53.841948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:53.841948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:53.841948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:53.842778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:53.842778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:53.842778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:53.842778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:53.842778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:53.842778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:53.842778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:53.842778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:53.842778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:53.842778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:53.842778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:52:53.842778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:53.842778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 05:52:53.842778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:53.856428 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 371/732] 10.222.0.20 () {56 vars in 1105 bytes} [Thu Oct 20 05:52:53 2022] GET /identity/v3/services/image => generated 86 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:53.857689 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a21f7b-ed01-4a3f-a268-f328d40e81d8 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:53.863859 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a1306-2d2f-43d3-a3a9-60d5e691fa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:53.864475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a1306-2d2f-43d3-a3a9-60d5e691fa2d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:53.864782 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a1306-2d2f-43d3-a3a9-60d5e691fa2d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:53.865058 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a1306-2d2f-43d3-a3a9-60d5e691fa2d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:53.865819 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a1306-2d2f-43d3-a3a9-60d5e691fa2d None admin] RBAC: Adding query filter params (interface=admin, service_id=2395e318c4ad4c4c9d0710a1c07f5245, region_id=RegionOne) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:53.866197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a1306-2d2f-43d3-a3a9-60d5e691fa2d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:53.867634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a1306-2d2f-43d3-a3a9-60d5e691fa2d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:53.878383 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 362/733] 10.222.0.20 () {56 vars in 1249 bytes} [Thu Oct 20 05:52:53 2022] GET /identity/v3/endpoints?service_id=2395e318c4ad4c4c9d0710a1c07f5245&interface=admin®ion_id=RegionOne => generated 193 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:53.879259 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a1306-2d2f-43d3-a3a9-60d5e691fa2d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:53.978993 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:53.998756 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a1306-2d2f-43d3-a3a9-60d5e691fa2d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:54.023478 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:54.038491 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df64fc32-6a07-4a60-b749-df002a0f3786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:54.039114 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df64fc32-6a07-4a60-b749-df002a0f3786 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:54.039452 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df64fc32-6a07-4a60-b749-df002a0f3786 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:54.039739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df64fc32-6a07-4a60-b749-df002a0f3786 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:54.040348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df64fc32-6a07-4a60-b749-df002a0f3786 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:54.041824 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df64fc32-6a07-4a60-b749-df002a0f3786 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:54.048961 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 363/734] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Oct 20 05:52:53 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 05:52:54.107781 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a21f7b-ed01-4a3f-a268-f328d40e81d8 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:54.149455 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac3f81-a697-475c-a56d-e559f9cc093c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:54.150084 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac3f81-a697-475c-a56d-e559f9cc093c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:54.150361 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac3f81-a697-475c-a56d-e559f9cc093c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:54.150629 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac3f81-a697-475c-a56d-e559f9cc093c None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:54.151348 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac3f81-a697-475c-a56d-e559f9cc093c None admin] RBAC: Adding query filter params (name=image) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:54.151643 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac3f81-a697-475c-a56d-e559f9cc093c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:54.153143 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac3f81-a697-475c-a56d-e559f9cc093c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:54.160095 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 372/735] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:54.166479 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df64fc32-6a07-4a60-b749-df002a0f3786 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:54.277468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df64fc32-6a07-4a60-b749-df002a0f3786 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:54.330883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3542ed4-f452-499a-b285-d0935bad7c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:54.331498 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3542ed4-f452-499a-b285-d0935bad7c0d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:54.331775 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3542ed4-f452-499a-b285-d0935bad7c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:54.332043 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3542ed4-f452-499a-b285-d0935bad7c0d None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:54.332779 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3542ed4-f452-499a-b285-d0935bad7c0d None admin] RBAC: Adding query filter params (type=image) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:54.333081 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3542ed4-f452-499a-b285-d0935bad7c0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:54.334490 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3542ed4-f452-499a-b285-d0935bad7c0d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:54.339633 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 364/736] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:52:54 2022] GET /identity/v3/services?type=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:54.569800 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e5378-4c40-4196-bf9d-4531980b73ef None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:54.570164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e5378-4c40-4196-bf9d-4531980b73ef None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:54.570270 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e5378-4c40-4196-bf9d-4531980b73ef None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:54.571830 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 373/737] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:54.577312 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e93398-8831-4192-bfad-46912bb226c9 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:54.577596 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e93398-8831-4192-bfad-46912bb226c9 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:54.577873 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e93398-8831-4192-bfad-46912bb226c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:54.597934 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a7e93398-8831-4192-bfad-46912bb226c9 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:54.599112 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a7e93398-8831-4192-bfad-46912bb226c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:54.636462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:54.671532 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 365/738] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 05:52:54.923299 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabfd16-491c-4f2e-8539-e870d7f38d37 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:54.924067 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabfd16-491c-4f2e-8539-e870d7f38d37 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:54.924578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabfd16-491c-4f2e-8539-e870d7f38d37 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:54.926427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 374/739] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:54.931886 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0d616-6146-41be-ae5e-bc5b3f8b57e8 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:54.932350 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0d616-6146-41be-ae5e-bc5b3f8b57e8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:54.932430 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0d616-6146-41be-ae5e-bc5b3f8b57e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:54.942022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aabfd16-491c-4f2e-8539-e870d7f38d37 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:54.952409 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-40d0d616-6146-41be-ae5e-bc5b3f8b57e8 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:54.953691 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-40d0d616-6146-41be-ae5e-bc5b3f8b57e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:55.057171 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 366/740] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:52:55.066630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:55.071692 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aabfd16-491c-4f2e-8539-e870d7f38d37 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:55.112659 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156015b-991c-445c-8f0e-74b487e0db37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:55.114080 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156015b-991c-445c-8f0e-74b487e0db37 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:55.114602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156015b-991c-445c-8f0e-74b487e0db37 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:55.115072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156015b-991c-445c-8f0e-74b487e0db37 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:55.115980 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156015b-991c-445c-8f0e-74b487e0db37 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:55.117616 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156015b-991c-445c-8f0e-74b487e0db37 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:55.135749 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 375/741] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Oct 20 05:52:54 2022] POST /identity/v3/services => generated 242 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:52:55.343367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0d616-6146-41be-ae5e-bc5b3f8b57e8 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:55.451265 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0d616-6146-41be-ae5e-bc5b3f8b57e8 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:55.453330 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:55.495928 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86627496-68de-42e9-a676-aea252272414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:55.496948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86627496-68de-42e9-a676-aea252272414 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:55.497431 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86627496-68de-42e9-a676-aea252272414 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:55.497903 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86627496-68de-42e9-a676-aea252272414 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:55.498840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86627496-68de-42e9-a676-aea252272414 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:55.500513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86627496-68de-42e9-a676-aea252272414 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:55.503461 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: WARNING keystone.server.flask.common [None req-86627496-68de-42e9-a676-aea252272414 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 20 05:52:55.523462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.sql.core [None req-86627496-68de-42e9-a676-aea252272414 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 20 05:52:55.523462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 20 05:52:55.523462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [parameters: {'id': '0ed48122bd004f0f991101179935ed30', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 20 05:52:55.523462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81354) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 20 05:52:55.532087 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-86627496-68de-42e9-a676-aea252272414 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0ed48122bd004f0f991101179935ed30.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0ed48122bd004f0f991101179935ed30. Oct 20 05:52:55.532087 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:55.532087 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 05:52:55.532087 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 05:52:55.532087 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 05:52:55.532087 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 05:52:55.532087 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 05:52:55.532087 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 05:52:55.532087 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 05:52:55.532087 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application conn.query(q) Oct 20 05:52:55.532087 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 05:52:55.532087 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 05:52:55.532087 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 05:52:55.533035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application result.read() Oct 20 05:52:55.533035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 05:52:55.533035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 05:52:55.533035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 05:52:55.533035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 05:52:55.533035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 05:52:55.533035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 05:52:55.533035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 05:52:55.533035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 05:52:55.533035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 20 05:52:55.533035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:55.533035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 20 05:52:55.533035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:55.533035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:55.533035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 20 05:52:55.533035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 20 05:52:55.533035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 20 05:52:55.533035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 20 05:52:55.534039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 05:52:55.534039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application next(self.gen) Oct 20 05:52:55.534039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 20 05:52:55.534039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application yield resource Oct 20 05:52:55.534039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 05:52:55.534039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application next(self.gen) Oct 20 05:52:55.534039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 20 05:52:55.534039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.session.rollback() Oct 20 05:52:55.534039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 20 05:52:55.534039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.force_reraise() Oct 20 05:52:55.534039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 20 05:52:55.534039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise self.value Oct 20 05:52:55.534039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 20 05:52:55.534039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 20 05:52:55.534039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 20 05:52:55.534039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application session.commit() Oct 20 05:52:55.534039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 20 05:52:55.534962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 20 05:52:55.534962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 20 05:52:55.534962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._prepare_impl() Oct 20 05:52:55.534962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 20 05:52:55.534962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.session.flush() Oct 20 05:52:55.534962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 20 05:52:55.534962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._flush(objects) Oct 20 05:52:55.534962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 20 05:52:55.534962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 20 05:52:55.534962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 20 05:52:55.534962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application compat.raise_( Oct 20 05:52:55.534962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 05:52:55.534962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception Oct 20 05:52:55.534962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 20 05:52:55.534962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application flush_context.execute() Oct 20 05:52:55.534962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 20 05:52:55.534962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rec.execute(self) Oct 20 05:52:55.536546 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 20 05:52:55.536546 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 20 05:52:55.536546 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 20 05:52:55.536546 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 20 05:52:55.536546 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 20 05:52:55.536546 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 20 05:52:55.536546 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 20 05:52:55.536546 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 20 05:52:55.536546 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 20 05:52:55.536546 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 20 05:52:55.536546 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 20 05:52:55.536546 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 20 05:52:55.536546 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 20 05:52:55.536546 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 20 05:52:55.536546 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 20 05:52:55.537369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 20 05:52:55.537369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 05:52:55.537369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception Oct 20 05:52:55.537369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 05:52:55.537369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 05:52:55.537369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 05:52:55.537369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 05:52:55.537369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 05:52:55.537369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 05:52:55.537369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 05:52:55.537369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application conn.query(q) Oct 20 05:52:55.537369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 05:52:55.537369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 05:52:55.537369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 05:52:55.537369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application result.read() Oct 20 05:52:55.537369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 05:52:55.537369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 05:52:55.538268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 05:52:55.538268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 05:52:55.538268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 05:52:55.538268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 05:52:55.538268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 05:52:55.538268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 05:52:55.538268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 20 05:52:55.538268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 20 05:52:55.538268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [parameters: {'id': '0ed48122bd004f0f991101179935ed30', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 20 05:52:55.538268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 20 05:52:55.538268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:55.538268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:52:55.538268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:55.538268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:55.538268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 20 05:52:55.539057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 20 05:52:55.539057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 20 05:52:55.539057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 20 05:52:55.539057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 20 05:52:55.539057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:55.539057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:52:55.539057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:55.539057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:55.539057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:55.539057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:55.539057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:55.539057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:55.539057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:55.539057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:55.539057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:55.539057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:55.539057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:55.540974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:55.540974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 20 05:52:55.540974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 20 05:52:55.540974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:55.540974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:55.540974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 20 05:52:55.540974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 20 05:52:55.540974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0ed48122bd004f0f991101179935ed30. Oct 20 05:52:55.540974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:55.543893 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 367/742] 10.222.0.20 () {60 vars in 1140 bytes} [Thu Oct 20 05:52:55 2022] POST /identity/v3/projects => generated 262 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:55.549731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156015b-991c-445c-8f0e-74b487e0db37 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:55.658270 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156015b-991c-445c-8f0e-74b487e0db37 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:55.694950 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a9701-c80c-49f5-a742-9f500c482848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:55.695588 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a9701-c80c-49f5-a742-9f500c482848 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:55.695863 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a9701-c80c-49f5-a742-9f500c482848 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:55.696131 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a9701-c80c-49f5-a742-9f500c482848 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:55.702924 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a9701-c80c-49f5-a742-9f500c482848 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:55.704934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a9701-c80c-49f5-a742-9f500c482848 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:55.716282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-679a9701-c80c-49f5-a742-9f500c482848 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:55.716282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:55.716282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:55.716282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:55.716282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:55.716282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:55.716282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:55.716282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:55.716282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:55.716282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:55.716282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:55.716282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:55.716282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:52:55.716282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:52:55.716282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:52:55.717224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:52:55.717224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:55.717224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:55.717224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:55.717224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:55.717224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:55.717224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:55.717224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:55.717224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:55.717224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:55.717224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:55.717224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:55.717224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:55.717224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:55.717224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:55.717224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:55.717224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:55.718127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:52:55.718127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:52:55.718127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:52:55.718127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:52:55.718127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:52:55.718127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:52:55.718127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:52:55.718127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:55.718127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 376/743] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:55.724265 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86627496-68de-42e9-a676-aea252272414 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:55.833131 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86627496-68de-42e9-a676-aea252272414 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:55.868240 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28ca21-0b0f-4555-913e-53e2e2192d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:55.868890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28ca21-0b0f-4555-913e-53e2e2192d07 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:55.869173 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28ca21-0b0f-4555-913e-53e2e2192d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:55.869447 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28ca21-0b0f-4555-913e-53e2e2192d07 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:55.870112 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28ca21-0b0f-4555-913e-53e2e2192d07 None admin] RBAC: Adding query filter params (name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:55.870415 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28ca21-0b0f-4555-913e-53e2e2192d07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:55.872101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28ca21-0b0f-4555-913e-53e2e2192d07 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:55.894422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 368/744] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Oct 20 05:52:55 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:56.369251 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee96494-6787-40dc-b01b-470ad5033742 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:56.370255 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee96494-6787-40dc-b01b-470ad5033742 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:56.370737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee96494-6787-40dc-b01b-470ad5033742 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:56.372542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 377/745] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:56.382188 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b0c1d-e747-48a1-9ce5-291935a8c926 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:56.382621 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b0c1d-e747-48a1-9ce5-291935a8c926 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:56.382797 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b0c1d-e747-48a1-9ce5-291935a8c926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:56.402801 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-4f7b0c1d-e747-48a1-9ce5-291935a8c926 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:56.404304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4f7b0c1d-e747-48a1-9ce5-291935a8c926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:56.415813 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:56.476583 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 369/746] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:56 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:52:56.633632 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce682e08-55b6-45a5-9726-db67e4c60a46 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:56.634123 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce682e08-55b6-45a5-9726-db67e4c60a46 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:56.634506 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce682e08-55b6-45a5-9726-db67e4c60a46 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:56.636421 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 378/747] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:56.641646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc70eb6-f710-452d-b1a8-27f6c7f94c4d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:56.643126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc70eb6-f710-452d-b1a8-27f6c7f94c4d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:56.643521 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc70eb6-f710-452d-b1a8-27f6c7f94c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:56.662306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-bfc70eb6-f710-452d-b1a8-27f6c7f94c4d None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:56.663489 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-bfc70eb6-f710-452d-b1a8-27f6c7f94c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:56.732931 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 370/748] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:56 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:52:56.745932 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce682e08-55b6-45a5-9726-db67e4c60a46 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:56.767569 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:56.860226 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce682e08-55b6-45a5-9726-db67e4c60a46 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:56.900304 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8cfcdc-6224-42cf-80b5-693eff869aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:56.900941 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cfcdc-6224-42cf-80b5-693eff869aba None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:56.901219 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cfcdc-6224-42cf-80b5-693eff869aba None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:56.901488 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cfcdc-6224-42cf-80b5-693eff869aba None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:56.902064 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8cfcdc-6224-42cf-80b5-693eff869aba None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:56.903425 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8cfcdc-6224-42cf-80b5-693eff869aba None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:56.912870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-3d8cfcdc-6224-42cf-80b5-693eff869aba None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 05:52:56.912870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:56.912870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:52:56.912870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:52:56.912870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:52:56.912870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:52:56.912870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:52:56.912870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:56.912870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 05:52:56.912870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:56.912870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:52:56.912870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:56.912870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:56.912870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:56.912870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:56.912870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:56.912870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:56.913836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:56.913836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:56.913836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:56.913836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:56.913836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:56.913836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:56.913836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:52:56.913836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:52:56.913836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:52:56.913836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:52:56.913836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:56.913836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:56.913836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:56.913836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:56.913836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:56.913836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:56.914750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:56.914750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:56.914750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:56.914750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:56.914750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:56.914750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:56.914750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:56.914750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:56.914750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:56.914750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:56.914750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:52:56.914750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:56.914750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 05:52:56.914750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:56.914750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 379/749] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Oct 20 05:52:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:56.921929 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc70eb6-f710-452d-b1a8-27f6c7f94c4d None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:57.001756 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:57.021901 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8cfcdc-6224-42cf-80b5-693eff869aba None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:57.033091 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc70eb6-f710-452d-b1a8-27f6c7f94c4d None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:57.070518 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015f31c-3ee2-4b74-90b8-cd5203281fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:57.071123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015f31c-3ee2-4b74-90b8-cd5203281fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:57.071395 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015f31c-3ee2-4b74-90b8-cd5203281fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:57.071762 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015f31c-3ee2-4b74-90b8-cd5203281fa8 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:57.072410 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015f31c-3ee2-4b74-90b8-cd5203281fa8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:57.072803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015f31c-3ee2-4b74-90b8-cd5203281fa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:57.074153 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015f31c-3ee2-4b74-90b8-cd5203281fa8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:57.079868 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 371/750] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Oct 20 05:52:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:57.085341 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015f31c-3ee2-4b74-90b8-cd5203281fa8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:57.110237 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:57.130354 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8cfcdc-6224-42cf-80b5-693eff869aba None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:57.176043 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb110b0c-6e93-4b7f-86d5-4fba32fab873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:57.176594 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb110b0c-6e93-4b7f-86d5-4fba32fab873 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:57.176907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb110b0c-6e93-4b7f-86d5-4fba32fab873 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:57.177181 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb110b0c-6e93-4b7f-86d5-4fba32fab873 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:57.177850 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb110b0c-6e93-4b7f-86d5-4fba32fab873 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:57.179431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb110b0c-6e93-4b7f-86d5-4fba32fab873 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:57.205622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 380/751] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Oct 20 05:52:57 2022] POST /identity/v3/services => generated 242 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 05:52:57.329000 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015f31c-3ee2-4b74-90b8-cd5203281fa8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:57.381904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06948cec-c356-4410-ae25-f3142ca23ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:57.382515 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06948cec-c356-4410-ae25-f3142ca23ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:57.383063 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06948cec-c356-4410-ae25-f3142ca23ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:57.383063 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06948cec-c356-4410-ae25-f3142ca23ed0 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:57.383634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06948cec-c356-4410-ae25-f3142ca23ed0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:57.383927 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06948cec-c356-4410-ae25-f3142ca23ed0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:57.385672 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06948cec-c356-4410-ae25-f3142ca23ed0 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:57.387082 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:57.393427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 372/752] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Oct 20 05:52:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:57.399614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb110b0c-6e93-4b7f-86d5-4fba32fab873 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:57.428947 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37f489-571f-4d67-ab76-875a7f818714 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:57.429214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37f489-571f-4d67-ab76-875a7f818714 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:57.429443 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37f489-571f-4d67-ab76-875a7f818714 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:57.430934 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 373/753] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:57.435892 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b1866-1c68-4b9e-8d1c-aa38a1a4ca26 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:57.436167 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b1866-1c68-4b9e-8d1c-aa38a1a4ca26 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:57.436439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b1866-1c68-4b9e-8d1c-aa38a1a4ca26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:57.461464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-c40b1866-1c68-4b9e-8d1c-aa38a1a4ca26 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:57.462931 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c40b1866-1c68-4b9e-8d1c-aa38a1a4ca26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:57.523999 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb110b0c-6e93-4b7f-86d5-4fba32fab873 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:57.556785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 374/754] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:57 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 05:52:57.569106 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7388f0a-12f6-436a-bb30-9dbf03788375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:57.569973 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7388f0a-12f6-436a-bb30-9dbf03788375 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:57.570438 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7388f0a-12f6-436a-bb30-9dbf03788375 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:57.570898 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7388f0a-12f6-436a-bb30-9dbf03788375 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:57.571755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7388f0a-12f6-436a-bb30-9dbf03788375 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:57.573345 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7388f0a-12f6-436a-bb30-9dbf03788375 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:57.594378 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 381/755] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 05:52:57 2022] POST /identity/v3/endpoints => generated 325 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:52:57.835804 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b1866-1c68-4b9e-8d1c-aa38a1a4ca26 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:57.939956 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b1866-1c68-4b9e-8d1c-aa38a1a4ca26 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:57.967417 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:57.982897 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb5c4b-4b53-446a-8bcc-0e51ee0a77be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:57.983531 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb5c4b-4b53-446a-8bcc-0e51ee0a77be None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:57.983805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb5c4b-4b53-446a-8bcc-0e51ee0a77be None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:57.984071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb5c4b-4b53-446a-8bcc-0e51ee0a77be None admin] PATH_INFO: `/v3/services/compute` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:57.984726 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb5c4b-4b53-446a-8bcc-0e51ee0a77be None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:57.986190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb5c4b-4b53-446a-8bcc-0e51ee0a77be None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:57.995197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-3adb5c4b-4b53-446a-8bcc-0e51ee0a77be None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 05:52:57.995197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:57.995197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:52:57.995197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:52:57.995197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:52:57.995197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:52:57.995197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:52:57.995197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:57.995197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 05:52:57.995197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:57.995197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:52:57.995197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:57.995197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:57.995197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:57.995197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:57.995197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:57.995197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:57.996285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:57.996285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:57.996285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:57.996285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:57.996285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:57.996285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:57.996285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:52:57.996285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:52:57.996285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:52:57.996285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:52:57.996285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:57.996285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:57.996285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:57.996285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:57.996285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:57.996285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:57.997254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:57.997254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:57.997254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:57.997254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:57.997254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:57.997254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:57.997254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:57.997254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:57.997254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:57.997254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:57.997254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:52:57.997254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:52:57.997254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 05:52:57.997254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:58.000817 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 375/756] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:52:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:58.003187 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7388f0a-12f6-436a-bb30-9dbf03788375 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:58.007240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:58.112895 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7388f0a-12f6-436a-bb30-9dbf03788375 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:58.151675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14796561-20e4-4c9e-a757-3f68bba1ccf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:58.152312 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14796561-20e4-4c9e-a757-3f68bba1ccf4 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:58.152652 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14796561-20e4-4c9e-a757-3f68bba1ccf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:58.152968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14796561-20e4-4c9e-a757-3f68bba1ccf4 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:58.153662 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14796561-20e4-4c9e-a757-3f68bba1ccf4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:58.153968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14796561-20e4-4c9e-a757-3f68bba1ccf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:58.155376 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14796561-20e4-4c9e-a757-3f68bba1ccf4 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:58.166427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 382/757] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Oct 20 05:52:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:58.168052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb5c4b-4b53-446a-8bcc-0e51ee0a77be None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:58.262106 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a01283-f061-47a1-aa45-7500f3be1533 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:58.262445 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a01283-f061-47a1-aa45-7500f3be1533 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:58.262652 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a01283-f061-47a1-aa45-7500f3be1533 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:58.264083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 383/758] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:52:58.270263 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa49d8-5e6d-4904-bd91-1d1f97e4e3fc None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:58.270800 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa49d8-5e6d-4904-bd91-1d1f97e4e3fc None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:58.271300 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa49d8-5e6d-4904-bd91-1d1f97e4e3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:58.281299 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb5c4b-4b53-446a-8bcc-0e51ee0a77be None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:58.291468 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f9aa49d8-5e6d-4904-bd91-1d1f97e4e3fc None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:58.293044 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f9aa49d8-5e6d-4904-bd91-1d1f97e4e3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:58.329901 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb17a9-f38f-496d-80a9-6a11dbcfd3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:58.331507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb17a9-f38f-496d-80a9-6a11dbcfd3ac None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:58.331995 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb17a9-f38f-496d-80a9-6a11dbcfd3ac None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:58.332482 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb17a9-f38f-496d-80a9-6a11dbcfd3ac None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:58.333531 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb17a9-f38f-496d-80a9-6a11dbcfd3ac None admin] RBAC: Adding query filter params (type=compute) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:58.334030 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb17a9-f38f-496d-80a9-6a11dbcfd3ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:58.335599 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb17a9-f38f-496d-80a9-6a11dbcfd3ac None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:58.341465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 376/759] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Oct 20 05:52:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:58.348206 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb17a9-f38f-496d-80a9-6a11dbcfd3ac None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:58.365631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:58.374126 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 384/760] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:58 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:52:58.408205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:58.460636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb17a9-f38f-496d-80a9-6a11dbcfd3ac None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:58.496509 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f5f07-931e-472a-baaa-515fd6dfc876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:58.497409 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f5f07-931e-472a-baaa-515fd6dfc876 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:58.497836 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f5f07-931e-472a-baaa-515fd6dfc876 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:58.498248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f5f07-931e-472a-baaa-515fd6dfc876 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:58.499078 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f5f07-931e-472a-baaa-515fd6dfc876 None admin] RBAC: Adding query filter params (interface=public, service_id=9a293f46c5ff43d3a03bae882827bf87, region_id=RegionOne) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:58.499528 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f5f07-931e-472a-baaa-515fd6dfc876 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:58.501044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f5f07-931e-472a-baaa-515fd6dfc876 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:58.510283 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 377/761] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Oct 20 05:52:58 2022] GET /identity/v3/endpoints?service_id=9a293f46c5ff43d3a03bae882827bf87&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:52:58.513206 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa49d8-5e6d-4904-bd91-1d1f97e4e3fc None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:58.619891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa49d8-5e6d-4904-bd91-1d1f97e4e3fc None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:58.656826 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e2448-2483-440e-ad24-674d37bc2f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:58.658375 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e2448-2483-440e-ad24-674d37bc2f62 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:58.658795 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e2448-2483-440e-ad24-674d37bc2f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:58.659214 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e2448-2483-440e-ad24-674d37bc2f62 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:58.659948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e2448-2483-440e-ad24-674d37bc2f62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:58.661470 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e2448-2483-440e-ad24-674d37bc2f62 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:58.672719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 385/762] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Oct 20 05:52:58 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 05:52:58.674254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f5f07-931e-472a-baaa-515fd6dfc876 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:58.790857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f5f07-931e-472a-baaa-515fd6dfc876 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:58.808962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:58.830292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410f27b-c12e-4a9e-8275-a4970d1f617b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:58.830999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410f27b-c12e-4a9e-8275-a4970d1f617b None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:58.831278 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410f27b-c12e-4a9e-8275-a4970d1f617b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:58.831545 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410f27b-c12e-4a9e-8275-a4970d1f617b None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:58.832245 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3410f27b-c12e-4a9e-8275-a4970d1f617b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:58.833740 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3410f27b-c12e-4a9e-8275-a4970d1f617b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:58.851853 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.sql.core [None req-3410f27b-c12e-4a9e-8275-a4970d1f617b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 20 05:52:58.851853 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 20 05:52:58.851853 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [parameters: {'user_id': '504c22c298e74abeaed24f9073ab4944', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 20 05:52:58.851853 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81354) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 20 05:52:58.859925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-3410f27b-c12e-4a9e-8275-a4970d1f617b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 20 05:52:58.859925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:58.859925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 05:52:58.859925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 05:52:58.859925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 05:52:58.859925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 05:52:58.859925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 05:52:58.859925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 05:52:58.859925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 05:52:58.859925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application conn.query(q) Oct 20 05:52:58.859925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 05:52:58.859925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 05:52:58.859925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 05:52:58.859925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application result.read() Oct 20 05:52:58.859925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 05:52:58.861149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 05:52:58.861149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 05:52:58.861149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 05:52:58.861149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 05:52:58.861149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 05:52:58.861149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 05:52:58.861149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 05:52:58.861149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 20 05:52:58.861149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:58.861149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 20 05:52:58.861149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:58.861149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:58.861149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 20 05:52:58.861149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 20 05:52:58.861149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 20 05:52:58.861149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 20 05:52:58.861149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 05:52:58.861149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application next(self.gen) Oct 20 05:52:58.862751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 20 05:52:58.862751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application yield resource Oct 20 05:52:58.862751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 05:52:58.862751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application next(self.gen) Oct 20 05:52:58.862751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 20 05:52:58.862751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.session.rollback() Oct 20 05:52:58.862751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 20 05:52:58.862751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.force_reraise() Oct 20 05:52:58.862751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 20 05:52:58.862751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise self.value Oct 20 05:52:58.862751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 20 05:52:58.862751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 20 05:52:58.862751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 20 05:52:58.862751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application session.commit() Oct 20 05:52:58.862751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 20 05:52:58.862751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 20 05:52:58.862751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 20 05:52:58.863890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._prepare_impl() Oct 20 05:52:58.863890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 20 05:52:58.863890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.session.flush() Oct 20 05:52:58.863890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 20 05:52:58.863890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._flush(objects) Oct 20 05:52:58.863890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 20 05:52:58.863890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 20 05:52:58.863890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 20 05:52:58.863890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application compat.raise_( Oct 20 05:52:58.863890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 05:52:58.863890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception Oct 20 05:52:58.863890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 20 05:52:58.863890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application flush_context.execute() Oct 20 05:52:58.863890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 20 05:52:58.863890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rec.execute(self) Oct 20 05:52:58.863890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 20 05:52:58.863890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 20 05:52:58.867957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 20 05:52:58.867957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 20 05:52:58.867957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 20 05:52:58.867957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 20 05:52:58.867957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 20 05:52:58.867957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 20 05:52:58.867957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 20 05:52:58.867957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 20 05:52:58.867957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 20 05:52:58.867957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 20 05:52:58.867957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 20 05:52:58.867957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 20 05:52:58.867957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 20 05:52:58.867957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 20 05:52:58.867957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 05:52:58.867957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception Oct 20 05:52:58.870408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 05:52:58.870408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 05:52:58.870408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 05:52:58.870408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 05:52:58.870408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 05:52:58.870408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 05:52:58.870408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 05:52:58.870408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application conn.query(q) Oct 20 05:52:58.870408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 05:52:58.870408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 05:52:58.870408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 05:52:58.870408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application result.read() Oct 20 05:52:58.870408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 05:52:58.870408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 05:52:58.870408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 05:52:58.870408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 05:52:58.872374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 05:52:58.872374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 05:52:58.872374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 05:52:58.872374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 05:52:58.872374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 20 05:52:58.872374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 20 05:52:58.872374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [parameters: {'user_id': '504c22c298e74abeaed24f9073ab4944', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 20 05:52:58.872374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 20 05:52:58.872374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:58.872374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:52:58.872374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:58.872374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:58.872374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:58.872374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:58.872374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:58.874359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:58.874359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:58.874359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:58.874359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:58.874359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:58.874359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:58.874359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:58.874359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 20 05:52:58.874359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 20 05:52:58.874359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:58.874359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:58.874359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:52:58.874359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:58.874359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:52:58.874359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:58.874359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 20 05:52:58.874359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 20 05:52:58.876786 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 20 05:52:58.876786 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 20 05:52:58.876786 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 20 05:52:58.876786 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 20 05:52:58.876786 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 20 05:52:58.876786 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:52:58.876786 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 378/763] 10.222.0.20 () {60 vars in 1131 bytes} [Thu Oct 20 05:52:58 2022] POST /identity/v3/users => generated 160 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:58.880781 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e2448-2483-440e-ad24-674d37bc2f62 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:58.942066 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:58.987239 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e2448-2483-440e-ad24-674d37bc2f62 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:59.023752 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ad105-edb4-46e7-af55-1d1b59f019ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:59.024490 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ad105-edb4-46e7-af55-1d1b59f019ec None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:59.024883 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ad105-edb4-46e7-af55-1d1b59f019ec None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:59.025220 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ad105-edb4-46e7-af55-1d1b59f019ec None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:59.034074 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ad105-edb4-46e7-af55-1d1b59f019ec None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:59.035611 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ad105-edb4-46e7-af55-1d1b59f019ec None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:59.047891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-c37ad105-edb4-46e7-af55-1d1b59f019ec None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 20 05:52:59.047891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:52:59.047891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:52:59.047891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:52:59.047891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:52:59.047891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:52:59.047891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:52:59.047891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:52:59.047891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:52:59.047891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:52:59.047891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:52:59.047891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:52:59.047891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:52:59.047891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:52:59.047891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:52:59.047891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:52:59.049163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:52:59.049163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:52:59.049163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:52:59.049163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:59.049163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:52:59.049163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:52:59.049163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:52:59.049163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:52:59.049163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:52:59.049163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:52:59.049163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:52:59.049163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:52:59.049163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:52:59.049163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:52:59.049163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:52:59.049163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:52:59.049163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:52:59.049163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:52:59.050309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:52:59.050309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:52:59.050309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:52:59.050309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:52:59.050309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:52:59.050309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:52:59.050309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:52:59.050309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:52:59.050309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 20 05:52:59.050309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:52:59.052277 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 386/764] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:52:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:52:59.057579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410f27b-c12e-4a9e-8275-a4970d1f617b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:59.163079 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410f27b-c12e-4a9e-8275-a4970d1f617b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:59.199064 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6775af4-53fe-4280-ad1b-ab010a838679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:52:59.200034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6775af4-53fe-4280-ad1b-ab010a838679 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:59.200506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6775af4-53fe-4280-ad1b-ab010a838679 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:59.201003 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6775af4-53fe-4280-ad1b-ab010a838679 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:59.201950 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6775af4-53fe-4280-ad1b-ab010a838679 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:52:59.202441 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6775af4-53fe-4280-ad1b-ab010a838679 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:52:59.204812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6775af4-53fe-4280-ad1b-ab010a838679 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:52:59.243741 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 379/765] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Oct 20 05:52:59 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:52:59.513871 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a4a6a-b773-4eee-aa29-2a4be968198a None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:59.514364 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a4a6a-b773-4eee-aa29-2a4be968198a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:59.514364 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a4a6a-b773-4eee-aa29-2a4be968198a None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:59.515778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 387/766] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:52:59.521796 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d2394-5313-43e4-9af8-58709e3afb79 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:59.522660 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d2394-5313-43e4-9af8-58709e3afb79 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:59.523151 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d2394-5313-43e4-9af8-58709e3afb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:59.542644 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-6a2d2394-5313-43e4-9af8-58709e3afb79 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:59.544499 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6a2d2394-5313-43e4-9af8-58709e3afb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:59.574867 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:52:59.616167 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 380/767] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:59 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:52:59.884538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a4a6a-b773-4eee-aa29-2a4be968198a None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:52:59.902688 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73689b6a-b94d-41fe-9e9d-f2bd46b3f85e None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:59.902976 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73689b6a-b94d-41fe-9e9d-f2bd46b3f85e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:59.903276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73689b6a-b94d-41fe-9e9d-f2bd46b3f85e None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:59.904746 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 381/768] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:52:59.910190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291a7b2-5703-4f24-a860-7d1c6494499d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:52:59.910474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291a7b2-5703-4f24-a860-7d1c6494499d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:52:59.910748 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291a7b2-5703-4f24-a860-7d1c6494499d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:52:59.929979 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-4291a7b2-5703-4f24-a860-7d1c6494499d None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:52:59.931168 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4291a7b2-5703-4f24-a860-7d1c6494499d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:52:59.995422 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a4a6a-b773-4eee-aa29-2a4be968198a None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:52:59.999777 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 382/769] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:52:59 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 05:53:00.015613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:00.030943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0017a-268d-4032-8b36-79d92a867c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:00.031659 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0017a-268d-4032-8b36-79d92a867c4b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:00.032005 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0017a-268d-4032-8b36-79d92a867c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:00.033043 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0017a-268d-4032-8b36-79d92a867c4b None admin] PATH_INFO: `/v3/services/image` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:00.033704 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b0017a-268d-4032-8b36-79d92a867c4b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:00.035184 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b0017a-268d-4032-8b36-79d92a867c4b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:00.044115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-16b0017a-268d-4032-8b36-79d92a867c4b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 05:53:00.044115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:00.044115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:53:00.044115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:53:00.044115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:53:00.044115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:53:00.044115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:53:00.044115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:53:00.044115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 05:53:00.044115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:53:00.044115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:53:00.044115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:53:00.044115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:00.044115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:53:00.044115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:53:00.044115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:53:00.044115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:53:00.045461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:53:00.045461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:53:00.045461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:53:00.045461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:53:00.045461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:53:00.045461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:53:00.045461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:53:00.045461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:53:00.045461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:53:00.045461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:53:00.045461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:53:00.045461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:53:00.045461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:53:00.045461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:53:00.045461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:53:00.045461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:53:00.046498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:53:00.046498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:53:00.046498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:53:00.046498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:53:00.046498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:53:00.046498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:53:00.046498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:53:00.046498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:53:00.046498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:53:00.046498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:53:00.046498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:53:00.046498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:53:00.046498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 05:53:00.046498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:53:00.049704 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 388/770] 10.222.0.20 () {56 vars in 1105 bytes} [Thu Oct 20 05:52:59 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:53:00.052413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291a7b2-5703-4f24-a860-7d1c6494499d None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:00.148583 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:00.154882 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291a7b2-5703-4f24-a860-7d1c6494499d None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:00.189470 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e153bd9-f55a-4d11-b479-aa74eed7eebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:00.190202 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e153bd9-f55a-4d11-b479-aa74eed7eebb None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:00.190545 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e153bd9-f55a-4d11-b479-aa74eed7eebb None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:00.190881 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e153bd9-f55a-4d11-b479-aa74eed7eebb None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:00.191566 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e153bd9-f55a-4d11-b479-aa74eed7eebb None admin] RBAC: Adding query filter params (name=image) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:00.191937 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e153bd9-f55a-4d11-b479-aa74eed7eebb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:00.193395 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e153bd9-f55a-4d11-b479-aa74eed7eebb None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:00.198417 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 383/771] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:53:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:00.203830 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0017a-268d-4032-8b36-79d92a867c4b None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:00.269873 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e153bd9-f55a-4d11-b479-aa74eed7eebb None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:00.311605 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0017a-268d-4032-8b36-79d92a867c4b None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:00.350970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ec29a-3071-4a2b-83b6-821feb2f9bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:00.351575 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ec29a-3071-4a2b-83b6-821feb2f9bab None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:00.351848 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ec29a-3071-4a2b-83b6-821feb2f9bab None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:00.352115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ec29a-3071-4a2b-83b6-821feb2f9bab None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:00.353022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3ec29a-3071-4a2b-83b6-821feb2f9bab None admin] RBAC: Adding query filter params (type=image) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:00.353165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3ec29a-3071-4a2b-83b6-821feb2f9bab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:00.354401 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3ec29a-3071-4a2b-83b6-821feb2f9bab None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:00.364157 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 389/772] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:53:00 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:00.365253 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ec29a-3071-4a2b-83b6-821feb2f9bab None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:00.391060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e153bd9-f55a-4d11-b479-aa74eed7eebb None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:00.414864 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:00.429051 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab279d39-b014-4850-b170-d063d921de3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:00.429811 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab279d39-b014-4850-b170-d063d921de3f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:00.430179 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab279d39-b014-4850-b170-d063d921de3f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:00.430574 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab279d39-b014-4850-b170-d063d921de3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:00.431717 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab279d39-b014-4850-b170-d063d921de3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:00.433264 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab279d39-b014-4850-b170-d063d921de3f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:00.435077 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 384/773] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:53:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:53:00.441252 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab279d39-b014-4850-b170-d063d921de3f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:00.472235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:00.474679 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ec29a-3071-4a2b-83b6-821feb2f9bab None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:00.512308 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633af40-10ca-4b65-8f61-ecbbf427d3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:00.513079 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633af40-10ca-4b65-8f61-ecbbf427d3d1 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:00.513413 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633af40-10ca-4b65-8f61-ecbbf427d3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:00.513745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633af40-10ca-4b65-8f61-ecbbf427d3d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:00.514536 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5633af40-10ca-4b65-8f61-ecbbf427d3d1 None admin] RBAC: Adding query filter params (interface=public, service_id=6c970d882a5842aa9a59073564139fed, region_id=RegionOne) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:00.515091 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5633af40-10ca-4b65-8f61-ecbbf427d3d1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:00.516331 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5633af40-10ca-4b65-8f61-ecbbf427d3d1 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:00.526976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 390/774] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Oct 20 05:53:00 2022] GET /identity/v3/endpoints?service_id=6c970d882a5842aa9a59073564139fed&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:53:00.527978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633af40-10ca-4b65-8f61-ecbbf427d3d1 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:00.553971 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab279d39-b014-4850-b170-d063d921de3f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:00.587977 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab4285-8a02-4a8a-acda-2142ab6a00b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:00.588583 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab4285-8a02-4a8a-acda-2142ab6a00b6 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:00.588887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab4285-8a02-4a8a-acda-2142ab6a00b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:00.589158 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab4285-8a02-4a8a-acda-2142ab6a00b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:00.589802 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab4285-8a02-4a8a-acda-2142ab6a00b6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:00.591219 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab4285-8a02-4a8a-acda-2142ab6a00b6 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:00.625529 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 385/775] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Oct 20 05:53:00 2022] POST /identity/v3/projects => generated 311 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 05:53:00.640232 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633af40-10ca-4b65-8f61-ecbbf427d3d1 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:00.656046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:00.681012 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01df1d-3aec-473b-ac60-da80625c2443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:00.681650 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01df1d-3aec-473b-ac60-da80625c2443 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:00.681932 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01df1d-3aec-473b-ac60-da80625c2443 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:00.682202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01df1d-3aec-473b-ac60-da80625c2443 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:00.682820 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd01df1d-3aec-473b-ac60-da80625c2443 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:00.684214 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd01df1d-3aec-473b-ac60-da80625c2443 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:00.698709 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 391/776] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Oct 20 05:53:00 2022] GET /identity/v3/services => generated 1957 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 05:53:00.977590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f393483-68cb-4f3f-8f42-e3b57aee1f03 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:00.977951 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f393483-68cb-4f3f-8f42-e3b57aee1f03 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:00.978054 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f393483-68cb-4f3f-8f42-e3b57aee1f03 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:00.979530 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 386/777] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:53:00.984663 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e185326-6d43-4a6e-b920-239b231f96ae None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:00.984985 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e185326-6d43-4a6e-b920-239b231f96ae None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:00.985259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e185326-6d43-4a6e-b920-239b231f96ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:01.003931 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-0e185326-6d43-4a6e-b920-239b231f96ae None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:53:01.005157 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0e185326-6d43-4a6e-b920-239b231f96ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:53:01.074394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 392/778] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:53:00 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:53:01.343468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f393483-68cb-4f3f-8f42-e3b57aee1f03 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:01.432247 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:01.456192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f393483-68cb-4f3f-8f42-e3b57aee1f03 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:01.490791 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa737fe3-491b-45b0-b68b-1d06a114d1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:01.491431 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa737fe3-491b-45b0-b68b-1d06a114d1aa None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:01.491709 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa737fe3-491b-45b0-b68b-1d06a114d1aa None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:01.491978 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa737fe3-491b-45b0-b68b-1d06a114d1aa None admin] PATH_INFO: `/v3/services/compute` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:01.492551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa737fe3-491b-45b0-b68b-1d06a114d1aa None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:01.493942 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa737fe3-491b-45b0-b68b-1d06a114d1aa None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:01.502781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-fa737fe3-491b-45b0-b68b-1d06a114d1aa None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 05:53:01.502781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:01.502781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:53:01.502781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:53:01.502781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:53:01.502781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:53:01.502781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:53:01.502781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:53:01.502781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 05:53:01.502781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:53:01.502781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:53:01.502781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:53:01.502781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:01.502781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:53:01.502781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:53:01.502781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:53:01.502781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:53:01.503783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:53:01.503783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:53:01.503783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:53:01.503783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:53:01.503783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:53:01.503783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:53:01.503783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:53:01.503783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:53:01.503783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:53:01.503783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:53:01.503783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:53:01.503783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:53:01.503783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:53:01.503783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:53:01.503783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:53:01.503783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:53:01.504647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:53:01.504647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:53:01.504647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:53:01.504647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:53:01.504647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:53:01.504647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:53:01.504647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:53:01.504647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:53:01.504647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:53:01.504647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:53:01.504647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:53:01.504647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:53:01.504647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 05:53:01.504647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:53:01.508790 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 387/779] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:53:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:53:01.510381 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e185326-6d43-4a6e-b920-239b231f96ae None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:01.613620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e185326-6d43-4a6e-b920-239b231f96ae None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:01.621440 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:01.650126 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed1ec2-a2d2-4e69-9ba2-e110cb6d543e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:01.650737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed1ec2-a2d2-4e69-9ba2-e110cb6d543e None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:01.651017 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed1ec2-a2d2-4e69-9ba2-e110cb6d543e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:01.651286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed1ec2-a2d2-4e69-9ba2-e110cb6d543e None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:01.651903 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed1ec2-a2d2-4e69-9ba2-e110cb6d543e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:01.652204 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed1ec2-a2d2-4e69-9ba2-e110cb6d543e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:01.653605 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed1ec2-a2d2-4e69-9ba2-e110cb6d543e None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:01.663932 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 393/780] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Oct 20 05:53:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:53:01.664837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa737fe3-491b-45b0-b68b-1d06a114d1aa None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:01.763231 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa737fe3-491b-45b0-b68b-1d06a114d1aa None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:01.796510 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d1faa-aee6-4dc3-a9ff-dec385f1bfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:01.797128 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d1faa-aee6-4dc3-a9ff-dec385f1bfaa None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:01.797404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d1faa-aee6-4dc3-a9ff-dec385f1bfaa None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:01.797670 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d1faa-aee6-4dc3-a9ff-dec385f1bfaa None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:01.798489 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d1faa-aee6-4dc3-a9ff-dec385f1bfaa None admin] RBAC: Adding query filter params (type=compute) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:01.798687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d1faa-aee6-4dc3-a9ff-dec385f1bfaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:01.800038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d1faa-aee6-4dc3-a9ff-dec385f1bfaa None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:01.811163 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 388/781] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Oct 20 05:53:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:53:01.811325 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed1ec2-a2d2-4e69-9ba2-e110cb6d543e None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:01.913401 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed1ec2-a2d2-4e69-9ba2-e110cb6d543e None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:01.949030 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bdab17-e1a1-454a-93a0-4a2a1627081c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:01.949642 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bdab17-e1a1-454a-93a0-4a2a1627081c None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:01.949921 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bdab17-e1a1-454a-93a0-4a2a1627081c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:01.950193 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bdab17-e1a1-454a-93a0-4a2a1627081c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:01.950863 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bdab17-e1a1-454a-93a0-4a2a1627081c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:01.952223 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bdab17-e1a1-454a-93a0-4a2a1627081c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:01.984736 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 394/782] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 05:53:01 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 05:53:02.866167 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d407a2-ce5e-492f-a04e-905b2f915117 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:02.866690 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d407a2-ce5e-492f-a04e-905b2f915117 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:02.866690 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d407a2-ce5e-492f-a04e-905b2f915117 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:02.868739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 389/783] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:53:02.873358 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69fec6-5ff5-44e8-bd70-94a9370a04a2 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:02.873790 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69fec6-5ff5-44e8-bd70-94a9370a04a2 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:02.873959 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69fec6-5ff5-44e8-bd70-94a9370a04a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:02.875418 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788946d7-a061-4675-ab05-5e1a96a653a4 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:02.875696 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788946d7-a061-4675-ab05-5e1a96a653a4 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:02.876599 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788946d7-a061-4675-ab05-5e1a96a653a4 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:02.877835 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 390/784] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:53:02.882933 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4763b-f92d-4f09-9adb-38c3c09a4570 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:02.883216 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4763b-f92d-4f09-9adb-38c3c09a4570 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:02.883485 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4763b-f92d-4f09-9adb-38c3c09a4570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:02.893949 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-bd69fec6-5ff5-44e8-bd70-94a9370a04a2 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:53:02.895130 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-bd69fec6-5ff5-44e8-bd70-94a9370a04a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:53:02.901435 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-46c4763b-f92d-4f09-9adb-38c3c09a4570 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:53:02.902600 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-46c4763b-f92d-4f09-9adb-38c3c09a4570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:53:02.904581 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:02.917482 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:02.978839 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 395/785] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:53:02 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:53:02.983443 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 391/786] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:53:02 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 05:53:03.252355 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69fec6-5ff5-44e8-bd70-94a9370a04a2 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:03.255851 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4763b-f92d-4f09-9adb-38c3c09a4570 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:03.362670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69fec6-5ff5-44e8-bd70-94a9370a04a2 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:03.370813 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4763b-f92d-4f09-9adb-38c3c09a4570 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:03.400405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30076e-367c-4d24-bcf2-54b0618a06c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:03.401167 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30076e-367c-4d24-bcf2-54b0618a06c0 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:03.401440 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30076e-367c-4d24-bcf2-54b0618a06c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:03.401713 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30076e-367c-4d24-bcf2-54b0618a06c0 None admin] PATH_INFO: `/v3/services/image` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:03.402277 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30076e-367c-4d24-bcf2-54b0618a06c0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:03.403743 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30076e-367c-4d24-bcf2-54b0618a06c0 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:03.405691 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b1c21-e35c-4349-affa-a7ff874fe357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:03.406327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b1c21-e35c-4349-affa-a7ff874fe357 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:03.406607 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b1c21-e35c-4349-affa-a7ff874fe357 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:03.406882 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b1c21-e35c-4349-affa-a7ff874fe357 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:03.408126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b1c21-e35c-4349-affa-a7ff874fe357 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:03.409577 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b1c21-e35c-4349-affa-a7ff874fe357 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:03.411153 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 392/787] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:53:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:53:03.413631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-4c30076e-367c-4d24-bcf2-54b0618a06c0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 05:53:03.413631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:03.413631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:53:03.413631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:53:03.413631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:53:03.413631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:53:03.413631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:53:03.413631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:53:03.413631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 05:53:03.413631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:53:03.413631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:53:03.413631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:53:03.413631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:03.413631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:53:03.413631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:53:03.413631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:53:03.413631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:53:03.414647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:53:03.414647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:53:03.414647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:53:03.414647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:53:03.414647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:53:03.414647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:53:03.414647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:53:03.414647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:53:03.414647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:53:03.414647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:53:03.414647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:53:03.414647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:53:03.414647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:53:03.414647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:53:03.414647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:53:03.414647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:53:03.415500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:53:03.415500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:53:03.415500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:53:03.415500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:53:03.415500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:53:03.415500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:53:03.415500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:53:03.415500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:53:03.415500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:53:03.415500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:53:03.415500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:53:03.415500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:53:03.415500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 05:53:03.415500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:53:03.415500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 396/788] 10.222.0.20 () {56 vars in 1105 bytes} [Thu Oct 20 05:53:03 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:53:03.421591 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30076e-367c-4d24-bcf2-54b0618a06c0 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:03.421737 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b1c21-e35c-4349-affa-a7ff874fe357 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:03.424858 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:03.441775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:03.525256 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b1c21-e35c-4349-affa-a7ff874fe357 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:03.537460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30076e-367c-4d24-bcf2-54b0618a06c0 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:03.558220 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdda00e-24a7-4aad-a5d7-6fb9590203dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:03.558819 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdda00e-24a7-4aad-a5d7-6fb9590203dc None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:03.559095 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdda00e-24a7-4aad-a5d7-6fb9590203dc None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:03.559362 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdda00e-24a7-4aad-a5d7-6fb9590203dc None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:03.559998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdda00e-24a7-4aad-a5d7-6fb9590203dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:03.561438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdda00e-24a7-4aad-a5d7-6fb9590203dc None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:03.576597 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13cba5-6e2f-4755-b913-dc164ed7cc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:03.577475 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13cba5-6e2f-4755-b913-dc164ed7cc37 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:03.578161 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13cba5-6e2f-4755-b913-dc164ed7cc37 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:03.578949 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13cba5-6e2f-4755-b913-dc164ed7cc37 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:03.580372 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f13cba5-6e2f-4755-b913-dc164ed7cc37 None admin] RBAC: Adding query filter params (name=image) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:03.581084 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f13cba5-6e2f-4755-b913-dc164ed7cc37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:03.582604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f13cba5-6e2f-4755-b913-dc164ed7cc37 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:03.589815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 393/789] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Oct 20 05:53:03 2022] POST /identity/v3/users => generated 285 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:53:03.590007 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 397/790] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:53:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:03.595745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdda00e-24a7-4aad-a5d7-6fb9590203dc None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:03.660964 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:03.709134 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdda00e-24a7-4aad-a5d7-6fb9590203dc None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:03.745402 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1d8f8-eefe-43fe-8677-3b0ca2f27b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:03.746020 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1d8f8-eefe-43fe-8677-3b0ca2f27b2a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:03.746294 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1d8f8-eefe-43fe-8677-3b0ca2f27b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:03.746563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1d8f8-eefe-43fe-8677-3b0ca2f27b2a None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:03.747220 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1d8f8-eefe-43fe-8677-3b0ca2f27b2a None admin] RBAC: Adding query filter params (type=image) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:03.747521 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1d8f8-eefe-43fe-8677-3b0ca2f27b2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:03.748947 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1d8f8-eefe-43fe-8677-3b0ca2f27b2a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:03.759261 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 394/791] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:53:03 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:53:03.761261 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13cba5-6e2f-4755-b913-dc164ed7cc37 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:03.850297 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:03.871192 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13cba5-6e2f-4755-b913-dc164ed7cc37 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:03.907279 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5a0f6-2e59-44ed-b90a-74da80ff0327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:03.908201 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5a0f6-2e59-44ed-b90a-74da80ff0327 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:03.908484 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5a0f6-2e59-44ed-b90a-74da80ff0327 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:03.908780 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5a0f6-2e59-44ed-b90a-74da80ff0327 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:03.909460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5a0f6-2e59-44ed-b90a-74da80ff0327 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:03.910816 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5a0f6-2e59-44ed-b90a-74da80ff0327 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:03.940724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 398/792] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 05:53:03 2022] POST /identity/v3/endpoints => generated 327 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 05:53:04.238707 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592dd0d9-9b11-4928-9fa8-10614864aa07 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:04.239264 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592dd0d9-9b11-4928-9fa8-10614864aa07 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:04.239264 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592dd0d9-9b11-4928-9fa8-10614864aa07 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:04.240744 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 395/793] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:53:04.245758 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2442c-1e9d-4d94-a740-de3a370a9d09 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:04.246139 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2442c-1e9d-4d94-a740-de3a370a9d09 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:04.246482 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2442c-1e9d-4d94-a740-de3a370a9d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:04.265901 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-10d2442c-1e9d-4d94-a740-de3a370a9d09 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:53:04.267377 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-10d2442c-1e9d-4d94-a740-de3a370a9d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:53:04.353198 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 399/794] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:53:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:53:04.623262 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592dd0d9-9b11-4928-9fa8-10614864aa07 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:04.726822 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592dd0d9-9b11-4928-9fa8-10614864aa07 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:04.749868 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:04.763539 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3036d87-34e0-4b00-a677-ed1bf1430046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:04.764172 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3036d87-34e0-4b00-a677-ed1bf1430046 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:04.764445 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3036d87-34e0-4b00-a677-ed1bf1430046 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:04.764726 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3036d87-34e0-4b00-a677-ed1bf1430046 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:04.775399 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3036d87-34e0-4b00-a677-ed1bf1430046 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:53:04.780139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-d3036d87-34e0-4b00-a677-ed1bf1430046 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 20 05:53:04.780139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:04.780139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:53:04.780139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:53:04.780139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:53:04.780139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:53:04.780139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:53:04.780139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:53:04.780139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:53:04.780139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:53:04.780139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:53:04.780139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:53:04.780139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:53:04.780139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:53:04.780139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:53:04.780139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:53:04.781317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:53:04.781317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:53:04.781317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:53:04.781317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:53:04.781317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:53:04.781317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:53:04.781317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:53:04.781317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:53:04.781317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:53:04.781317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:53:04.781317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:53:04.781317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:53:04.781317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:53:04.781317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:53:04.781317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:53:04.781317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:53:04.782267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:53:04.782267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:53:04.782267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:53:04.782267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:53:04.782267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:53:04.782267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:53:04.782267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:53:04.782267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:53:04.782267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:53:04.782267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:53:04.782267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 20 05:53:04.782267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:53:04.783660 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 396/795] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Oct 20 05:53:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:53:04.787420 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2442c-1e9d-4d94-a740-de3a370a9d09 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:04.851662 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:04.893143 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2442c-1e9d-4d94-a740-de3a370a9d09 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:04.926386 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622c148-6a01-4d76-8484-bfcb66e8992c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:04.927018 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622c148-6a01-4d76-8484-bfcb66e8992c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:04.927240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622c148-6a01-4d76-8484-bfcb66e8992c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:04.927555 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622c148-6a01-4d76-8484-bfcb66e8992c None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:04.928152 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5622c148-6a01-4d76-8484-bfcb66e8992c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:04.928445 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5622c148-6a01-4d76-8484-bfcb66e8992c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:04.929835 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5622c148-6a01-4d76-8484-bfcb66e8992c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:04.938024 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 400/796] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Oct 20 05:53:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:04.943632 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3036d87-34e0-4b00-a677-ed1bf1430046 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:05.042170 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3036d87-34e0-4b00-a677-ed1bf1430046 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:05.075152 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665cd044-176d-45d6-ac22-c1142cfd65cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:05.075769 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665cd044-176d-45d6-ac22-c1142cfd65cb None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:05.076042 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665cd044-176d-45d6-ac22-c1142cfd65cb None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:05.076368 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665cd044-176d-45d6-ac22-c1142cfd65cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:05.086713 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665cd044-176d-45d6-ac22-c1142cfd65cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:05.088118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665cd044-176d-45d6-ac22-c1142cfd65cb None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:05.102496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-665cd044-176d-45d6-ac22-c1142cfd65cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:53:05.102496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:05.102496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:53:05.102496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:53:05.102496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:53:05.102496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:53:05.102496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:53:05.102496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:53:05.102496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:53:05.102496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:53:05.102496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:53:05.102496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:53:05.102496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:53:05.102496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:53:05.102496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:53:05.102496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:53:05.103464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:53:05.103464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:53:05.103464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:53:05.103464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:53:05.103464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:53:05.103464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:53:05.103464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:53:05.103464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:53:05.103464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:53:05.103464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:53:05.103464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:53:05.103464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:53:05.103464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:53:05.103464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:53:05.103464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:53:05.103464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:53:05.103464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:53:05.104442 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:53:05.104442 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:53:05.104442 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:53:05.105209 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 397/797] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:53:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:53:05.109275 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622c148-6a01-4d76-8484-bfcb66e8992c None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:05.213621 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622c148-6a01-4d76-8484-bfcb66e8992c None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:05.237394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:05.249420 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e919b-8d19-483e-9165-dd7e41abc341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:05.250040 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e919b-8d19-483e-9165-dd7e41abc341 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:05.250322 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e919b-8d19-483e-9165-dd7e41abc341 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:05.250596 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e919b-8d19-483e-9165-dd7e41abc341 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:05.251303 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e919b-8d19-483e-9165-dd7e41abc341 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:05.251608 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e919b-8d19-483e-9165-dd7e41abc341 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:05.253022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e919b-8d19-483e-9165-dd7e41abc341 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:05.269951 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 401/798] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:53:05.271861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665cd044-176d-45d6-ac22-c1142cfd65cb None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:05.314557 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:05.381982 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665cd044-176d-45d6-ac22-c1142cfd65cb None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:05.415747 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43b909-fbbe-48a1-b5f4-b92b3b2bffb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:05.416367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43b909-fbbe-48a1-b5f4-b92b3b2bffb6 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:05.416643 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43b909-fbbe-48a1-b5f4-b92b3b2bffb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:05.416936 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43b909-fbbe-48a1-b5f4-b92b3b2bffb6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:05.423865 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43b909-fbbe-48a1-b5f4-b92b3b2bffb6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:05.425324 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43b909-fbbe-48a1-b5f4-b92b3b2bffb6 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:05.436475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-7a43b909-fbbe-48a1-b5f4-b92b3b2bffb6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 20 05:53:05.436475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:05.436475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:53:05.436475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:53:05.436475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:53:05.436475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:53:05.436475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:53:05.436475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:53:05.436475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:53:05.436475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:53:05.436475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:53:05.436475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:53:05.436475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:53:05.436475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:53:05.436475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:53:05.436475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:53:05.437437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:53:05.437437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:53:05.437437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:53:05.437437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:53:05.437437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:53:05.437437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:53:05.437437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:53:05.437437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:53:05.437437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:53:05.437437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:53:05.437437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:53:05.437437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:53:05.437437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:53:05.437437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:53:05.437437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:53:05.437437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:53:05.437437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:53:05.437437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:53:05.438900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:53:05.438900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:53:05.438900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:53:05.438900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:53:05.438900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:53:05.438900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:53:05.438900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:53:05.438900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:53:05.438900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 20 05:53:05.438900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:53:05.440767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 398/799] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Oct 20 05:53:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:53:05.443910 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e919b-8d19-483e-9165-dd7e41abc341 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:05.546040 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e919b-8d19-483e-9165-dd7e41abc341 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:05.579893 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a9848-4d83-45a1-b1a8-f5d9b62f9891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:05.580505 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a9848-4d83-45a1-b1a8-f5d9b62f9891 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:05.580821 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a9848-4d83-45a1-b1a8-f5d9b62f9891 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:05.581095 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a9848-4d83-45a1-b1a8-f5d9b62f9891 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:05.581820 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a9848-4d83-45a1-b1a8-f5d9b62f9891 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:05.582125 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a9848-4d83-45a1-b1a8-f5d9b62f9891 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:05.583543 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a9848-4d83-45a1-b1a8-f5d9b62f9891 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:05.605642 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 402/800] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Oct 20 05:53:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:05.606968 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43b909-fbbe-48a1-b5f4-b92b3b2bffb6 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:05.703408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:05.712531 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43b909-fbbe-48a1-b5f4-b92b3b2bffb6 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:05.749004 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e46d8-de9a-4936-8154-b3e824057eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:05.749793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e46d8-de9a-4936-8154-b3e824057eea None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:05.750174 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e46d8-de9a-4936-8154-b3e824057eea None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:05.750533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e46d8-de9a-4936-8154-b3e824057eea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:05.761531 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e46d8-de9a-4936-8154-b3e824057eea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:05.763086 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e46d8-de9a-4936-8154-b3e824057eea None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:05.777560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-fb5e46d8-de9a-4936-8154-b3e824057eea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:53:05.777560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:05.777560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:53:05.777560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:53:05.777560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:53:05.777560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:53:05.777560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:53:05.777560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:53:05.777560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:53:05.777560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:53:05.777560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:53:05.777560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:53:05.777560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:53:05.777560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:53:05.777560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:53:05.777560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:53:05.778712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:53:05.778712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:53:05.778712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:53:05.778712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:53:05.778712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:53:05.778712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:53:05.778712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:53:05.778712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:53:05.778712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:53:05.778712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:53:05.778712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:53:05.778712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:53:05.778712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:53:05.778712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:53:05.778712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:53:05.778712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:53:05.778712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:53:05.779810 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:53:05.779810 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:53:05.779810 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:53:05.784726 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 399/801] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:53:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:53:05.787525 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a9848-4d83-45a1-b1a8-f5d9b62f9891 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:05.819247 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41c980-947b-4244-9ad3-b5259548e764 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:05.819452 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41c980-947b-4244-9ad3-b5259548e764 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:05.819730 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41c980-947b-4244-9ad3-b5259548e764 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:05.821228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 400/802] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:53:05.825838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee39d27-e5d4-4189-9e79-5ff456d98d45 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:05.826117 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee39d27-e5d4-4189-9e79-5ff456d98d45 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:05.826389 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee39d27-e5d4-4189-9e79-5ff456d98d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:05.835052 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:05.847073 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-9ee39d27-e5d4-4189-9e79-5ff456d98d45 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:53:05.848262 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9ee39d27-e5d4-4189-9e79-5ff456d98d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:53:05.901938 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a9848-4d83-45a1-b1a8-f5d9b62f9891 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:05.922263 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 401/803] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:53:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 05:53:05.939800 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48600a7-4c76-4149-904a-46962c146b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:05.940471 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48600a7-4c76-4149-904a-46962c146b15 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:05.940783 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48600a7-4c76-4149-904a-46962c146b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:05.941016 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48600a7-4c76-4149-904a-46962c146b15 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:05.941690 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48600a7-4c76-4149-904a-46962c146b15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:05.941985 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48600a7-4c76-4149-904a-46962c146b15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:05.943343 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48600a7-4c76-4149-904a-46962c146b15 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:05.955524 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 403/804] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:53:05.960951 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee39d27-e5d4-4189-9e79-5ff456d98d45 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:06.043214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:06.067926 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee39d27-e5d4-4189-9e79-5ff456d98d45 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:06.103579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4282a-12df-40c3-9a8e-c6c75926c097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:06.104439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4282a-12df-40c3-9a8e-c6c75926c097 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:06.104894 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4282a-12df-40c3-9a8e-c6c75926c097 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:06.105312 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4282a-12df-40c3-9a8e-c6c75926c097 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:06.112439 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75535256-cf91-4e82-a3fd-a23d15b79370 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:06.112749 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75535256-cf91-4e82-a3fd-a23d15b79370 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:06.112935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c4282a-12df-40c3-9a8e-c6c75926c097 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:06.113048 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75535256-cf91-4e82-a3fd-a23d15b79370 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:06.114467 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 404/805] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:53:06.114571 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c4282a-12df-40c3-9a8e-c6c75926c097 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:06.119367 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e654f4e-1d6f-401f-b378-312ea62817b9 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:06.119650 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e654f4e-1d6f-401f-b378-312ea62817b9 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:06.119922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e654f4e-1d6f-401f-b378-312ea62817b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:06.126101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-d7c4282a-12df-40c3-9a8e-c6c75926c097 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:53:06.126101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:06.126101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:53:06.126101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:53:06.126101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:53:06.126101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:53:06.126101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:53:06.126101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:53:06.126101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:53:06.126101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:53:06.126101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:53:06.126101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:53:06.126101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:53:06.126101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:53:06.126101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:53:06.127098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:53:06.127098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:53:06.127098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:53:06.127098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:53:06.127098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:53:06.127098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:53:06.127098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:53:06.127098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:53:06.127098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:53:06.127098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:53:06.127098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:53:06.127098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:53:06.127098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:53:06.127098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:53:06.127098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:53:06.127098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:53:06.127098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:53:06.128508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:53:06.128508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:53:06.128508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:53:06.128508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:53:06.128508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:53:06.128508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:53:06.128508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:53:06.128508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:53:06.128508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 402/806] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:53:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:53:06.132672 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4282a-12df-40c3-9a8e-c6c75926c097 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:06.143963 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-0e654f4e-1d6f-401f-b378-312ea62817b9 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:53:06.145553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0e654f4e-1d6f-401f-b378-312ea62817b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:53:06.217588 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 405/807] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:53:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 05:53:06.217588 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e654f4e-1d6f-401f-b378-312ea62817b9 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:06.236323 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4282a-12df-40c3-9a8e-c6c75926c097 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:06.256715 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:06.271664 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b5557-f7c8-4430-8501-564bf43603b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:06.272052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b5557-f7c8-4430-8501-564bf43603b0 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:06.272588 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b5557-f7c8-4430-8501-564bf43603b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:06.272588 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b5557-f7c8-4430-8501-564bf43603b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:06.273283 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b5557-f7c8-4430-8501-564bf43603b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:06.273584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b5557-f7c8-4430-8501-564bf43603b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:06.274990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b5557-f7c8-4430-8501-564bf43603b0 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:06.277603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:06.287861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 403/808] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Oct 20 05:53:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:53:06.293515 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b5557-f7c8-4430-8501-564bf43603b0 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:06.348127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e654f4e-1d6f-401f-b378-312ea62817b9 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:06.387378 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd4cc1-259b-48ae-a0bd-3b3339a1b20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:06.388268 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd4cc1-259b-48ae-a0bd-3b3339a1b20f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:06.388549 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd4cc1-259b-48ae-a0bd-3b3339a1b20f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:06.388855 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd4cc1-259b-48ae-a0bd-3b3339a1b20f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:06.399329 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b5557-f7c8-4430-8501-564bf43603b0 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:06.402159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd4cc1-259b-48ae-a0bd-3b3339a1b20f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:53:06.407404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-63bd4cc1-259b-48ae-a0bd-3b3339a1b20f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:53:06.407404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:06.407404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:53:06.407404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:53:06.407404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:53:06.407404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:53:06.407404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:53:06.407404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:53:06.407404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:53:06.407404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:53:06.407404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:53:06.407404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:53:06.407404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:53:06.407404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:53:06.407404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:53:06.407404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:53:06.408352 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:53:06.408352 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:53:06.408352 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:53:06.408352 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:53:06.408352 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:53:06.408352 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:53:06.408352 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:53:06.408352 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:53:06.408352 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:53:06.408352 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:53:06.408352 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:53:06.408352 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:53:06.408352 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:53:06.408352 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:53:06.408352 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:53:06.408352 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:53:06.409345 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:53:06.409345 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:53:06.409345 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:53:06.409345 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:53:06.409345 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:53:06.409345 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:53:06.409345 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:53:06.409345 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:53:06.409345 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:53:06.409345 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:53:06.409345 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:53:06.409345 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:53:06.411351 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 406/809] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Oct 20 05:53:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:53:06.417158 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd4cc1-259b-48ae-a0bd-3b3339a1b20f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:06.435540 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f1c54-9a1e-44d3-8623-a92cf8e7bb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:06.436199 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f1c54-9a1e-44d3-8623-a92cf8e7bb57 None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:06.436483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f1c54-9a1e-44d3-8623-a92cf8e7bb57 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:06.436837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f1c54-9a1e-44d3-8623-a92cf8e7bb57 None admin] PATH_INFO: `/v3/projects/0ee65b9a8229495bbe97318f060d6342/users/ae2c1c886cd94d9f90082f66e7e8c9f4/roles/b53e713d4b4a460b9465321b090b0455` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:06.439646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f1c54-9a1e-44d3-8623-a92cf8e7bb57 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ee65b9a8229495bbe97318f060d6342, user_id=ae2c1c886cd94d9f90082f66e7e8c9f4, role_id=b53e713d4b4a460b9465321b090b0455)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:06.441095 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f1c54-9a1e-44d3-8623-a92cf8e7bb57 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:06.456346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 404/810] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:53:06 2022] PUT /identity/v3/projects/0ee65b9a8229495bbe97318f060d6342/users/ae2c1c886cd94d9f90082f66e7e8c9f4/roles/b53e713d4b4a460b9465321b090b0455 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 05:53:06.494516 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f1c54-9a1e-44d3-8623-a92cf8e7bb57 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:06.526862 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:06.539117 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd4cc1-259b-48ae-a0bd-3b3339a1b20f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:06.560815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:06.583071 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d53b7-ecad-4c60-ae77-0935147882d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:06.583701 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d53b7-ecad-4c60-ae77-0935147882d0 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:06.584429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d53b7-ecad-4c60-ae77-0935147882d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:06.584429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d53b7-ecad-4c60-ae77-0935147882d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:06.584930 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d53b7-ecad-4c60-ae77-0935147882d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:06.585234 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d53b7-ecad-4c60-ae77-0935147882d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:06.586679 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d53b7-ecad-4c60-ae77-0935147882d0 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:06.596602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 407/811] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:53:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:53:06.602171 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d53b7-ecad-4c60-ae77-0935147882d0 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:06.618353 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f1c54-9a1e-44d3-8623-a92cf8e7bb57 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:06.670930 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9d50b-4579-47ac-99c5-f047e443597b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:06.671463 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9d50b-4579-47ac-99c5-f047e443597b None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:06.671876 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9d50b-4579-47ac-99c5-f047e443597b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:06.672150 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9d50b-4579-47ac-99c5-f047e443597b None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:06.673764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f9d50b-4579-47ac-99c5-f047e443597b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:06.675377 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f9d50b-4579-47ac-99c5-f047e443597b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:06.684888 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.sql.core [None req-71f9d50b-4579-47ac-99c5-f047e443597b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 20 05:53:06.684888 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 20 05:53:06.684888 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [parameters: {'id': 'd02d3c8429c34cc486df6ca6f6628eae', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 20 05:53:06.684888 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81354) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 20 05:53:06.695396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-71f9d50b-4579-47ac-99c5-f047e443597b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 20 05:53:06.695396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:06.695396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 05:53:06.695396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 05:53:06.695396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 05:53:06.695396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 05:53:06.695396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 05:53:06.695396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 05:53:06.695396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 05:53:06.695396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application conn.query(q) Oct 20 05:53:06.695396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 05:53:06.695396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 05:53:06.695396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 05:53:06.695396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application result.read() Oct 20 05:53:06.696394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 05:53:06.696394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 05:53:06.696394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 05:53:06.696394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 05:53:06.696394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 05:53:06.696394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 05:53:06.696394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 05:53:06.696394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 05:53:06.696394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 20 05:53:06.696394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:53:06.696394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 20 05:53:06.696394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:53:06.696394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:06.696394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 20 05:53:06.696394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 20 05:53:06.696394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 20 05:53:06.696394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 20 05:53:06.696394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 05:53:06.702422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application next(self.gen) Oct 20 05:53:06.702422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 20 05:53:06.702422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application yield resource Oct 20 05:53:06.702422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 05:53:06.702422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application next(self.gen) Oct 20 05:53:06.702422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 20 05:53:06.702422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.session.rollback() Oct 20 05:53:06.702422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 20 05:53:06.702422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.force_reraise() Oct 20 05:53:06.702422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 20 05:53:06.702422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise self.value Oct 20 05:53:06.702422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 20 05:53:06.702422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 20 05:53:06.702422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 20 05:53:06.702422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application session.commit() Oct 20 05:53:06.702422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 20 05:53:06.702422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 20 05:53:06.703462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 20 05:53:06.703462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._prepare_impl() Oct 20 05:53:06.703462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 20 05:53:06.703462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.session.flush() Oct 20 05:53:06.703462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 20 05:53:06.703462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._flush(objects) Oct 20 05:53:06.703462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 20 05:53:06.703462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 20 05:53:06.703462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 20 05:53:06.703462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application compat.raise_( Oct 20 05:53:06.703462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 05:53:06.703462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception Oct 20 05:53:06.703462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 20 05:53:06.703462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application flush_context.execute() Oct 20 05:53:06.703462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 20 05:53:06.703462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rec.execute(self) Oct 20 05:53:06.703462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 20 05:53:06.705098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 20 05:53:06.705098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 20 05:53:06.705098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 20 05:53:06.705098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 20 05:53:06.705098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 20 05:53:06.705098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 20 05:53:06.705098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 20 05:53:06.705098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 20 05:53:06.705098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 20 05:53:06.705098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 20 05:53:06.705098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 20 05:53:06.705098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 20 05:53:06.705098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 20 05:53:06.705098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 20 05:53:06.705098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 20 05:53:06.706111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 05:53:06.706111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception Oct 20 05:53:06.706111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 05:53:06.706111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 05:53:06.706111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 05:53:06.706111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 05:53:06.706111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 05:53:06.706111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 05:53:06.706111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 05:53:06.706111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application conn.query(q) Oct 20 05:53:06.706111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 05:53:06.706111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 05:53:06.706111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 05:53:06.706111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application result.read() Oct 20 05:53:06.706111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 05:53:06.706111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 05:53:06.706111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 05:53:06.707197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 05:53:06.707197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 05:53:06.707197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 05:53:06.707197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 05:53:06.707197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 05:53:06.707197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 20 05:53:06.707197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 20 05:53:06.707197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [parameters: {'id': 'd02d3c8429c34cc486df6ca6f6628eae', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 20 05:53:06.707197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 20 05:53:06.707197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:53:06.707197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:53:06.707197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:53:06.707197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:06.707197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 20 05:53:06.707197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 20 05:53:06.708138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 20 05:53:06.708138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 20 05:53:06.708138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 20 05:53:06.708138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:53:06.708138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:53:06.708138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:53:06.708138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:06.708138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:53:06.708138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:53:06.708138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:53:06.708138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:53:06.708138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:53:06.708138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:53:06.708138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:53:06.708138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:53:06.708138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:53:06.708138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:53:06.709280 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 20 05:53:06.709280 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 20 05:53:06.709280 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:53:06.709280 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:53:06.709280 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 20 05:53:06.709280 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 20 05:53:06.709280 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:53:06.709280 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:53:06.709280 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 20 05:53:06.709280 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 20 05:53:06.709280 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 20 05:53:06.709280 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:53:06.709280 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 405/812] 10.222.0.20 () {60 vars in 1137 bytes} [Thu Oct 20 05:53:06 2022] POST /identity/v3/domains => generated 190 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:53:06.711995 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9d50b-4579-47ac-99c5-f047e443597b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:06.713837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d53b7-ecad-4c60-ae77-0935147882d0 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:06.750097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f40665-3bdb-4f69-84bd-2e59f417b56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:06.750972 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f40665-3bdb-4f69-84bd-2e59f417b56b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:06.751264 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f40665-3bdb-4f69-84bd-2e59f417b56b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:06.751535 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f40665-3bdb-4f69-84bd-2e59f417b56b None admin] PATH_INFO: `/v3/users/4fa16a6d840c42b780889d271db8c2b8` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:06.765182 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f40665-3bdb-4f69-84bd-2e59f417b56b None admin] RBAC: Authorizing `identity:get_user(user_id=4fa16a6d840c42b780889d271db8c2b8)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:06.767152 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f40665-3bdb-4f69-84bd-2e59f417b56b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:06.768796 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 408/813] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:53:06 2022] GET /identity/v3/users/4fa16a6d840c42b780889d271db8c2b8 => generated 285 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:53:06.774720 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f40665-3bdb-4f69-84bd-2e59f417b56b None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:06.790790 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:06.828166 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9d50b-4579-47ac-99c5-f047e443597b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:06.851275 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:06.867685 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173bd9c-7c07-4058-85e3-f823c8bc5de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:06.868433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173bd9c-7c07-4058-85e3-f823c8bc5de8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:06.868847 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173bd9c-7c07-4058-85e3-f823c8bc5de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:06.869073 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173bd9c-7c07-4058-85e3-f823c8bc5de8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:06.879168 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f40665-3bdb-4f69-84bd-2e59f417b56b None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:06.880734 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173bd9c-7c07-4058-85e3-f823c8bc5de8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:06.882341 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173bd9c-7c07-4058-85e3-f823c8bc5de8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:06.896612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-3173bd9c-7c07-4058-85e3-f823c8bc5de8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:53:06.896612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:06.896612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:53:06.896612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:53:06.896612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:53:06.896612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:53:06.896612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:53:06.896612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:53:06.896612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:53:06.896612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:53:06.896612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:53:06.896612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:53:06.896612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:53:06.896612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:53:06.896612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:53:06.896612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:53:06.897805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:53:06.897805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:53:06.897805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:53:06.897805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:53:06.897805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:53:06.897805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:53:06.897805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:53:06.897805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:53:06.897805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:53:06.897805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:53:06.897805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:53:06.897805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:53:06.897805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:53:06.897805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:53:06.897805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:53:06.897805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:53:06.897805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:53:06.898964 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:53:06.898964 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:53:06.898964 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:53:06.902680 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 406/814] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:53:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:53:06.904249 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173bd9c-7c07-4058-85e3-f823c8bc5de8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:06.918740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3b1a3-2b26-409b-b216-bace45d92a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:06.919454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3b1a3-2b26-409b-b216-bace45d92a37 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:06.919796 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3b1a3-2b26-409b-b216-bace45d92a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:06.920062 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3b1a3-2b26-409b-b216-bace45d92a37 None admin] PATH_INFO: `/v3/projects/f768d5001c4d42a9ad55bb5dfa58ef62` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:06.921372 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3b1a3-2b26-409b-b216-bace45d92a37 None admin] RBAC: Authorizing `identity:get_project(project_id=f768d5001c4d42a9ad55bb5dfa58ef62)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:06.922874 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3b1a3-2b26-409b-b216-bace45d92a37 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:06.929041 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 409/815] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:53:06 2022] GET /identity/v3/projects/f768d5001c4d42a9ad55bb5dfa58ef62 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:06.930628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3b1a3-2b26-409b-b216-bace45d92a37 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:07.012410 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173bd9c-7c07-4058-85e3-f823c8bc5de8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:07.017600 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:07.038865 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3b1a3-2b26-409b-b216-bace45d92a37 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:07.047111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43419c94-5ee4-4629-9144-c079eaa159a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:07.047906 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43419c94-5ee4-4629-9144-c079eaa159a6 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:07.048195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43419c94-5ee4-4629-9144-c079eaa159a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:07.048465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43419c94-5ee4-4629-9144-c079eaa159a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:07.049141 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43419c94-5ee4-4629-9144-c079eaa159a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:07.049443 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43419c94-5ee4-4629-9144-c079eaa159a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:07.050803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43419c94-5ee4-4629-9144-c079eaa159a6 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:07.062748 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 407/816] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:53:07.076038 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd95d87-cdad-4b46-8b1b-ebc075c5de0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:07.076763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd95d87-cdad-4b46-8b1b-ebc075c5de0f None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:07.077041 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd95d87-cdad-4b46-8b1b-ebc075c5de0f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:07.077309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd95d87-cdad-4b46-8b1b-ebc075c5de0f None admin] PATH_INFO: `/v3/projects/f768d5001c4d42a9ad55bb5dfa58ef62/users/4fa16a6d840c42b780889d271db8c2b8/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:07.079404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd95d87-cdad-4b46-8b1b-ebc075c5de0f None admin] RBAC: Authorizing `identity:create_grant(project_id=f768d5001c4d42a9ad55bb5dfa58ef62, user_id=4fa16a6d840c42b780889d271db8c2b8, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:07.080902 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd95d87-cdad-4b46-8b1b-ebc075c5de0f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:07.096766 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 410/817] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:53:06 2022] PUT /identity/v3/projects/f768d5001c4d42a9ad55bb5dfa58ef62/users/4fa16a6d840c42b780889d271db8c2b8/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 05:53:08.620747 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba56284-5b6a-4021-b1c5-c668dd5edcb3 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:08.621250 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba56284-5b6a-4021-b1c5-c668dd5edcb3 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:08.621250 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba56284-5b6a-4021-b1c5-c668dd5edcb3 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:08.622538 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 408/818] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:53:08.628391 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534a082-811f-4311-9845-1828afc07b9d None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:08.628842 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534a082-811f-4311-9845-1828afc07b9d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:08.629115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534a082-811f-4311-9845-1828afc07b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:08.648609 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-5534a082-811f-4311-9845-1828afc07b9d None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:53:08.649815 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5534a082-811f-4311-9845-1828afc07b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:53:08.715422 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 411/819] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:53:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:53:08.991057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba56284-5b6a-4021-b1c5-c668dd5edcb3 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:09.022463 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:09.091685 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba56284-5b6a-4021-b1c5-c668dd5edcb3 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:09.125060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0eb12-cd0a-4238-8c5e-7eb7c24ef27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:09.125766 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0eb12-cd0a-4238-8c5e-7eb7c24ef27c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:09.126041 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0eb12-cd0a-4238-8c5e-7eb7c24ef27c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:09.126390 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0eb12-cd0a-4238-8c5e-7eb7c24ef27c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:09.138309 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0eb12-cd0a-4238-8c5e-7eb7c24ef27c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:53:09.143483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-9be0eb12-cd0a-4238-8c5e-7eb7c24ef27c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 20 05:53:09.143483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:09.143483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:53:09.143483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:53:09.143483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:53:09.143483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:53:09.143483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:53:09.143483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:53:09.143483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:53:09.143483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:53:09.143483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:53:09.143483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:53:09.143483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:53:09.143483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:53:09.143483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:53:09.143483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:53:09.144732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:53:09.144732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:53:09.144732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:53:09.144732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:53:09.144732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:53:09.144732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:53:09.144732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:53:09.144732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:53:09.144732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:53:09.144732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:53:09.144732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:53:09.144732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:53:09.144732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:53:09.144732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:53:09.144732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:53:09.144732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:53:09.145799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:53:09.145799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:53:09.145799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:53:09.145799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:53:09.145799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:53:09.145799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:53:09.145799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:53:09.145799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:53:09.145799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:53:09.145799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:53:09.145799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 20 05:53:09.145799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:53:09.148901 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 409/820] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Oct 20 05:53:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:53:09.150855 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534a082-811f-4311-9845-1828afc07b9d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:09.218764 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:09.243276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219c853-639f-48c4-8669-c9974b7c43ea None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:09.243485 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219c853-639f-48c4-8669-c9974b7c43ea None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:09.243753 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219c853-639f-48c4-8669-c9974b7c43ea None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:09.245698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 410/821] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:53:09.250728 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b08a0-e29d-4054-9def-e9107ba2c017 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:09.251012 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b08a0-e29d-4054-9def-e9107ba2c017 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:09.251197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534a082-811f-4311-9845-1828afc07b9d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:09.251297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b08a0-e29d-4054-9def-e9107ba2c017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:09.268816 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a84b08a0-e29d-4054-9def-e9107ba2c017 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:53:09.270002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a84b08a0-e29d-4054-9def-e9107ba2c017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:53:09.285882 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f3d69-a3b0-4898-bef1-44fc4c7c68ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:09.286484 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f3d69-a3b0-4898-bef1-44fc4c7c68ac None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:09.286756 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f3d69-a3b0-4898-bef1-44fc4c7c68ac None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:09.287025 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f3d69-a3b0-4898-bef1-44fc4c7c68ac None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:09.287672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f3d69-a3b0-4898-bef1-44fc4c7c68ac None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:09.287977 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f3d69-a3b0-4898-bef1-44fc4c7c68ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:09.289378 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f3d69-a3b0-4898-bef1-44fc4c7c68ac None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:09.303287 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 412/822] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Oct 20 05:53:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:09.304368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f3d69-a3b0-4898-bef1-44fc4c7c68ac None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:09.342003 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 411/823] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:53:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 05:53:09.404197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f3d69-a3b0-4898-bef1-44fc4c7c68ac None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:09.433658 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:09.439281 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f45549d-5a65-422f-97a7-8e78974df813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:09.439911 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45549d-5a65-422f-97a7-8e78974df813 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:09.440216 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45549d-5a65-422f-97a7-8e78974df813 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:09.440492 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45549d-5a65-422f-97a7-8e78974df813 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:09.451153 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f45549d-5a65-422f-97a7-8e78974df813 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:09.452581 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f45549d-5a65-422f-97a7-8e78974df813 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:09.467333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-5f45549d-5a65-422f-97a7-8e78974df813 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:53:09.467333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:09.467333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:53:09.467333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:53:09.467333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:53:09.467333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:53:09.467333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:53:09.467333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:53:09.467333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:53:09.467333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:53:09.467333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:53:09.467333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:53:09.467333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:53:09.467333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:53:09.467333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:53:09.467333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:53:09.468227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:53:09.468227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:53:09.468227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:53:09.468227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:53:09.468227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:53:09.468227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:53:09.468227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:53:09.468227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:53:09.468227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:53:09.468227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:53:09.468227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:53:09.468227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:53:09.468227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:53:09.468227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:53:09.468227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:53:09.468227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:53:09.468227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:53:09.469163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:53:09.469163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:53:09.469163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:53:09.469163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 413/824] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:53:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:53:09.474260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b08a0-e29d-4054-9def-e9107ba2c017 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:09.497843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:09.570964 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b08a0-e29d-4054-9def-e9107ba2c017 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:09.603060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdff9f9-df6a-4f7c-93ed-519948018cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:09.603650 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdff9f9-df6a-4f7c-93ed-519948018cae None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:09.603921 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdff9f9-df6a-4f7c-93ed-519948018cae None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:09.604185 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdff9f9-df6a-4f7c-93ed-519948018cae None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:09.604840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdff9f9-df6a-4f7c-93ed-519948018cae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:09.605137 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdff9f9-df6a-4f7c-93ed-519948018cae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:09.606486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdff9f9-df6a-4f7c-93ed-519948018cae None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:09.611324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f45549d-5a65-422f-97a7-8e78974df813 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:09.618816 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 412/825] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:53:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:09.624195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdff9f9-df6a-4f7c-93ed-519948018cae None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:09.704122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:09.712138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f45549d-5a65-422f-97a7-8e78974df813 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:09.721588 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdff9f9-df6a-4f7c-93ed-519948018cae None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:09.747301 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99838d-13c2-4311-8463-d0e4763af597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:09.747955 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99838d-13c2-4311-8463-d0e4763af597 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:09.748182 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99838d-13c2-4311-8463-d0e4763af597 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:09.748446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99838d-13c2-4311-8463-d0e4763af597 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:09.755773 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede518a9-b1f4-45d9-9fd0-cb6f88f71f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:09.756394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede518a9-b1f4-45d9-9fd0-cb6f88f71f0e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:09.756678 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede518a9-b1f4-45d9-9fd0-cb6f88f71f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:09.756976 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede518a9-b1f4-45d9-9fd0-cb6f88f71f0e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:09.759222 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99838d-13c2-4311-8463-d0e4763af597 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:53:09.764518 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede518a9-b1f4-45d9-9fd0-cb6f88f71f0e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:09.764675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-fa99838d-13c2-4311-8463-d0e4763af597 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:53:09.764675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:09.764675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:53:09.764675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:53:09.764675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:53:09.764675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:53:09.764675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:53:09.764675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:53:09.764675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:53:09.764675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:53:09.764675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:53:09.764675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:53:09.764675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:53:09.764675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:53:09.764675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:53:09.764675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:53:09.765809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:53:09.765809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:53:09.765809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:53:09.765809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:53:09.765809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:53:09.765809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:53:09.765809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:53:09.765809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:53:09.765809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:53:09.765809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:53:09.765809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:53:09.765809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:53:09.765809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:53:09.765809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:53:09.765809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:53:09.765809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:53:09.767015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:53:09.767015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:53:09.767015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:53:09.767015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:53:09.767015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:53:09.767015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:53:09.767015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:53:09.767015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:53:09.767015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:53:09.767015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:53:09.767015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:53:09.767015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:53:09.767015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 414/826] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Oct 20 05:53:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:53:09.767736 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede518a9-b1f4-45d9-9fd0-cb6f88f71f0e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:09.771581 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99838d-13c2-4311-8463-d0e4763af597 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:09.777842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-ede518a9-b1f4-45d9-9fd0-cb6f88f71f0e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 20 05:53:09.777842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:09.777842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:53:09.777842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:53:09.777842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:53:09.777842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:53:09.777842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:53:09.777842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:53:09.777842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:53:09.777842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:53:09.777842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:53:09.777842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:53:09.777842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:53:09.777842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:53:09.777842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:53:09.777842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:53:09.784307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:53:09.784307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:53:09.784307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:53:09.784307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:53:09.784307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:53:09.784307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:53:09.784307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:53:09.784307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:53:09.784307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:53:09.784307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:53:09.784307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:53:09.784307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:53:09.784307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:53:09.784307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:53:09.784307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:53:09.784307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:53:09.784307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:53:09.784307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:53:09.785750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:53:09.785750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:53:09.785750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:53:09.785750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:53:09.785750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:53:09.785750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:53:09.785750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:53:09.785750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:53:09.785750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 20 05:53:09.785750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:53:09.785750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 413/827] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Oct 20 05:53:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:53:09.785750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede518a9-b1f4-45d9-9fd0-cb6f88f71f0e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:09.817322 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:09.870298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99838d-13c2-4311-8463-d0e4763af597 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:09.881724 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede518a9-b1f4-45d9-9fd0-cb6f88f71f0e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:09.904142 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8750c7d-4480-465e-889c-3a41845beed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:09.904757 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8750c7d-4480-465e-889c-3a41845beed9 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:09.905032 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8750c7d-4480-465e-889c-3a41845beed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:09.905295 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8750c7d-4480-465e-889c-3a41845beed9 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:09.905938 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8750c7d-4480-465e-889c-3a41845beed9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:09.906234 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8750c7d-4480-465e-889c-3a41845beed9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:09.908175 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8750c7d-4480-465e-889c-3a41845beed9 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:09.914827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441fec3a-6af3-4e4b-8de3-a896316e8d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:09.915416 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fec3a-6af3-4e4b-8de3-a896316e8d2e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:09.915715 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fec3a-6af3-4e4b-8de3-a896316e8d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:09.915981 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fec3a-6af3-4e4b-8de3-a896316e8d2e None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:09.916712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441fec3a-6af3-4e4b-8de3-a896316e8d2e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:09.917016 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441fec3a-6af3-4e4b-8de3-a896316e8d2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:09.917338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 415/828] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:53:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:53:09.918417 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441fec3a-6af3-4e4b-8de3-a896316e8d2e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:09.923206 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8750c7d-4480-465e-889c-3a41845beed9 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:09.934880 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:09.939078 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 414/829] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Oct 20 05:53:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:53:09.944555 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441fec3a-6af3-4e4b-8de3-a896316e8d2e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:10.020734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8750c7d-4480-465e-889c-3a41845beed9 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:10.027730 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:10.040675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441fec3a-6af3-4e4b-8de3-a896316e8d2e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:10.056113 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109e8e6-e371-494e-9877-1bbb18955b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:10.056766 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109e8e6-e371-494e-9877-1bbb18955b3d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:10.057157 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109e8e6-e371-494e-9877-1bbb18955b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:10.057429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109e8e6-e371-494e-9877-1bbb18955b3d None admin] PATH_INFO: `/v3/users/4fa16a6d840c42b780889d271db8c2b8` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:10.058541 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109e8e6-e371-494e-9877-1bbb18955b3d None admin] RBAC: Authorizing `identity:get_user(user_id=4fa16a6d840c42b780889d271db8c2b8)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:10.059966 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109e8e6-e371-494e-9877-1bbb18955b3d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:10.061535 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 416/830] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:53:09 2022] GET /identity/v3/users/4fa16a6d840c42b780889d271db8c2b8 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:53:10.067726 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109e8e6-e371-494e-9877-1bbb18955b3d None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:10.077935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854135b-b0e3-40bb-add3-0ecb432059fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:10.078539 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854135b-b0e3-40bb-add3-0ecb432059fc None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:10.079818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854135b-b0e3-40bb-add3-0ecb432059fc None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:10.080118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854135b-b0e3-40bb-add3-0ecb432059fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:10.145719 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:10.148044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0854135b-b0e3-40bb-add3-0ecb432059fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:10.149626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0854135b-b0e3-40bb-add3-0ecb432059fc None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:10.164987 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109e8e6-e371-494e-9877-1bbb18955b3d None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:10.198862 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3c356-c368-441b-b929-6035bb930fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:10.199457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3c356-c368-441b-b929-6035bb930fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:10.199732 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3c356-c368-441b-b929-6035bb930fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:10.199997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3c356-c368-441b-b929-6035bb930fb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:10.241784 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:10.248012 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 415/831] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:53:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 20 05:53:10.253802 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854135b-b0e3-40bb-add3-0ecb432059fc None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:10.267401 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab3c356-c368-441b-b929-6035bb930fb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:10.269015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab3c356-c368-441b-b929-6035bb930fb8 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:10.353053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:10.356414 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854135b-b0e3-40bb-add3-0ecb432059fc None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:10.367974 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 417/832] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:53:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 20 05:53:10.373776 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3c356-c368-441b-b929-6035bb930fb8 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:10.392400 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a918363-7564-4d90-9996-cb5cab460e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:10.393054 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a918363-7564-4d90-9996-cb5cab460e00 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:10.393328 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a918363-7564-4d90-9996-cb5cab460e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:10.393621 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a918363-7564-4d90-9996-cb5cab460e00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:10.403528 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a918363-7564-4d90-9996-cb5cab460e00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:10.404979 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a918363-7564-4d90-9996-cb5cab460e00 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:10.419686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-0a918363-7564-4d90-9996-cb5cab460e00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:53:10.419686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:10.419686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:53:10.419686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:53:10.419686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:53:10.419686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:53:10.419686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:53:10.419686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:53:10.419686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:53:10.419686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:53:10.419686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:53:10.419686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:53:10.419686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:53:10.419686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:53:10.419686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:53:10.419686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:53:10.420837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:53:10.420837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:53:10.420837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:53:10.420837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:53:10.420837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:53:10.420837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:53:10.420837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:53:10.420837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:53:10.420837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:53:10.420837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:53:10.420837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:53:10.420837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:53:10.420837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:53:10.420837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:53:10.420837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:53:10.420837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:53:10.420837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:53:10.421899 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:53:10.421899 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:53:10.421899 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:53:10.421899 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 416/833] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:53:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:53:10.426401 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a918363-7564-4d90-9996-cb5cab460e00 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:10.446796 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:10.473039 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3c356-c368-441b-b929-6035bb930fb8 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:10.507639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb21ac-cc2c-4ef6-8857-01c1a8da4da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:10.508269 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb21ac-cc2c-4ef6-8857-01c1a8da4da0 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:10.508548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb21ac-cc2c-4ef6-8857-01c1a8da4da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:10.508851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb21ac-cc2c-4ef6-8857-01c1a8da4da0 None admin] PATH_INFO: `/v3/projects/f768d5001c4d42a9ad55bb5dfa58ef62` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:10.510210 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb21ac-cc2c-4ef6-8857-01c1a8da4da0 None admin] RBAC: Authorizing `identity:get_project(project_id=f768d5001c4d42a9ad55bb5dfa58ef62)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:10.511636 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb21ac-cc2c-4ef6-8857-01c1a8da4da0 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:10.513286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 418/834] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:53:10 2022] GET /identity/v3/projects/f768d5001c4d42a9ad55bb5dfa58ef62 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:10.519271 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb21ac-cc2c-4ef6-8857-01c1a8da4da0 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:10.525042 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a918363-7564-4d90-9996-cb5cab460e00 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:10.559510 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f348609-3362-4b96-97f7-3da18afe9f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:10.560042 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f348609-3362-4b96-97f7-3da18afe9f5d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:10.560317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f348609-3362-4b96-97f7-3da18afe9f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:10.560584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f348609-3362-4b96-97f7-3da18afe9f5d None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:10.561252 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f348609-3362-4b96-97f7-3da18afe9f5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:10.561551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f348609-3362-4b96-97f7-3da18afe9f5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:10.562934 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f348609-3362-4b96-97f7-3da18afe9f5d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:10.574713 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 417/835] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:53:10.580204 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f348609-3362-4b96-97f7-3da18afe9f5d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:10.594044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:10.621264 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb21ac-cc2c-4ef6-8857-01c1a8da4da0 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:10.654165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf6da3-51b9-4ad4-a541-fe03173c90e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:10.654764 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf6da3-51b9-4ad4-a541-fe03173c90e2 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:10.655046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf6da3-51b9-4ad4-a541-fe03173c90e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:10.655317 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf6da3-51b9-4ad4-a541-fe03173c90e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:10.656022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf6da3-51b9-4ad4-a541-fe03173c90e2 None admin] RBAC: Adding query filter params (role.id=d8debbb193834fac9cc25f81726f3a67, scope.project.id=f768d5001c4d42a9ad55bb5dfa58ef62, user.id=4fa16a6d840c42b780889d271db8c2b8) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:10.656328 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf6da3-51b9-4ad4-a541-fe03173c90e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:10.658589 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf6da3-51b9-4ad4-a541-fe03173c90e2 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:10.664403 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 419/836] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:53:10 2022] GET /identity/v3/role_assignments?user.id=4fa16a6d840c42b780889d271db8c2b8&scope.project.id=f768d5001c4d42a9ad55bb5dfa58ef62&role.id=d8debbb193834fac9cc25f81726f3a67 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:53:10.679059 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f348609-3362-4b96-97f7-3da18afe9f5d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:10.715684 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df9003-e857-4f05-9cdb-3ed63fcb537c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:10.716340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df9003-e857-4f05-9cdb-3ed63fcb537c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:10.716618 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df9003-e857-4f05-9cdb-3ed63fcb537c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:10.716919 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df9003-e857-4f05-9cdb-3ed63fcb537c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:10.725193 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df9003-e857-4f05-9cdb-3ed63fcb537c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:10.726674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df9003-e857-4f05-9cdb-3ed63fcb537c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:10.739420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-f6df9003-e857-4f05-9cdb-3ed63fcb537c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:53:10.739420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:10.739420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:53:10.739420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:53:10.739420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:53:10.739420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:53:10.739420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:53:10.739420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:53:10.739420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:53:10.739420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:53:10.739420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:53:10.739420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:53:10.739420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:53:10.739420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:53:10.739420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:53:10.740637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:53:10.740637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:53:10.740637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:53:10.740637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:53:10.740637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:53:10.740637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:53:10.740637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:53:10.740637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:53:10.740637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:53:10.740637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:53:10.740637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:53:10.740637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:53:10.740637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:53:10.740637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:53:10.740637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:53:10.740637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:53:10.740637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:53:10.741566 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:53:10.741566 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:53:10.741566 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:53:10.741566 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:53:10.741566 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:53:10.741566 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:53:10.741566 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:53:10.741566 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:53:10.741566 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 418/837] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:53:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:53:10.746650 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf6da3-51b9-4ad4-a541-fe03173c90e2 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:10.756208 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:10.854357 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf6da3-51b9-4ad4-a541-fe03173c90e2 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:10.887985 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1954e4-f59d-4d2d-9ca3-999a9c98f093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:10.888595 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1954e4-f59d-4d2d-9ca3-999a9c98f093 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:10.888902 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1954e4-f59d-4d2d-9ca3-999a9c98f093 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:10.889173 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1954e4-f59d-4d2d-9ca3-999a9c98f093 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:10.889858 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1954e4-f59d-4d2d-9ca3-999a9c98f093 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:10.890161 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1954e4-f59d-4d2d-9ca3-999a9c98f093 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:10.892114 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1954e4-f59d-4d2d-9ca3-999a9c98f093 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:10.903916 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 420/838] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Oct 20 05:53:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:53:10.910596 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df9003-e857-4f05-9cdb-3ed63fcb537c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:10.979699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:11.011104 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df9003-e857-4f05-9cdb-3ed63fcb537c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:11.044519 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1ac4d-2899-4e3c-b3d0-2c0a49dd943d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:11.045145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1ac4d-2899-4e3c-b3d0-2c0a49dd943d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:11.045422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1ac4d-2899-4e3c-b3d0-2c0a49dd943d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:11.045689 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1ac4d-2899-4e3c-b3d0-2c0a49dd943d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:11.046400 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1ac4d-2899-4e3c-b3d0-2c0a49dd943d None admin] RBAC: Adding query filter params (role.id=b53e713d4b4a460b9465321b090b0455, scope.project.id=0ee65b9a8229495bbe97318f060d6342, user.id=ae2c1c886cd94d9f90082f66e7e8c9f4) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:11.046705 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1ac4d-2899-4e3c-b3d0-2c0a49dd943d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:11.048237 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1ac4d-2899-4e3c-b3d0-2c0a49dd943d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:11.054018 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 419/839] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:53:10 2022] GET /identity/v3/role_assignments?user.id=ae2c1c886cd94d9f90082f66e7e8c9f4&scope.project.id=0ee65b9a8229495bbe97318f060d6342&role.id=b53e713d4b4a460b9465321b090b0455 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:12.866680 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb0b49-660b-4c5c-acef-7b5727224ebe None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:12.867210 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb0b49-660b-4c5c-acef-7b5727224ebe None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:12.867210 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb0b49-660b-4c5c-acef-7b5727224ebe None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:12.868523 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 421/840] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:53:12.873693 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9cc4a-5de2-4653-b649-8eb3f6c41285 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:12.874095 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9cc4a-5de2-4653-b649-8eb3f6c41285 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:12.874239 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9cc4a-5de2-4653-b649-8eb3f6c41285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:12.895229 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-fee9cc4a-5de2-4653-b649-8eb3f6c41285 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:53:12.896402 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-fee9cc4a-5de2-4653-b649-8eb3f6c41285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:53:12.962276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 420/841] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:53:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:53:13.240390 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb0b49-660b-4c5c-acef-7b5727224ebe None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:13.306779 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:13.341567 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb0b49-660b-4c5c-acef-7b5727224ebe None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:13.374106 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a07ce-3bf1-4e52-901c-2d94900d08d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:13.374733 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a07ce-3bf1-4e52-901c-2d94900d08d2 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:13.375013 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a07ce-3bf1-4e52-901c-2d94900d08d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:13.375283 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a07ce-3bf1-4e52-901c-2d94900d08d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:13.377206 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a07ce-3bf1-4e52-901c-2d94900d08d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:13.378628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a07ce-3bf1-4e52-901c-2d94900d08d2 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:13.380473 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 422/842] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:53:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:13.386437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9cc4a-5de2-4653-b649-8eb3f6c41285 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:13.447701 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:13.481795 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9cc4a-5de2-4653-b649-8eb3f6c41285 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:13.514188 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553eec4-615a-439d-8659-1c86eb99d105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:13.514728 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553eec4-615a-439d-8659-1c86eb99d105 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:13.515006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553eec4-615a-439d-8659-1c86eb99d105 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:13.515278 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553eec4-615a-439d-8659-1c86eb99d105 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:13.515921 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a553eec4-615a-439d-8659-1c86eb99d105 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:13.518016 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a553eec4-615a-439d-8659-1c86eb99d105 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:13.539361 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 421/843] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Oct 20 05:53:13 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:53:15.690922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fff61-e816-4780-a78b-d1b828dc29c0 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:15.691463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fff61-e816-4780-a78b-d1b828dc29c0 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:15.691463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fff61-e816-4780-a78b-d1b828dc29c0 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:15.692812 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 423/844] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:53:15.697961 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5b9cb-13da-455d-98dd-804ad3a43ae5 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:15.698374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5b9cb-13da-455d-98dd-804ad3a43ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:15.698497 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5b9cb-13da-455d-98dd-804ad3a43ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:15.716036 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-fcf5b9cb-13da-455d-98dd-804ad3a43ae5 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:53:15.717236 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-fcf5b9cb-13da-455d-98dd-804ad3a43ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:53:15.780487 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 422/845] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:53:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:53:16.053314 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fff61-e816-4780-a78b-d1b828dc29c0 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:16.150801 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fff61-e816-4780-a78b-d1b828dc29c0 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:16.183604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c10b0d-f5d8-46d3-9637-d83b2e2bab03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:16.184222 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c10b0d-f5d8-46d3-9637-d83b2e2bab03 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:16.184498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c10b0d-f5d8-46d3-9637-d83b2e2bab03 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:16.184797 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c10b0d-f5d8-46d3-9637-d83b2e2bab03 None admin] PATH_INFO: `/v3/roles/49dae0405c764c96b7bfbcf9d9efd011` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:16.186170 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c10b0d-f5d8-46d3-9637-d83b2e2bab03 None admin] RBAC: Authorizing `identity:get_role(role_id=49dae0405c764c96b7bfbcf9d9efd011)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:16.187521 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c10b0d-f5d8-46d3-9637-d83b2e2bab03 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:16.189536 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 424/846] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:53:16 2022] GET /identity/v3/roles/49dae0405c764c96b7bfbcf9d9efd011 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:16.195299 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5b9cb-13da-455d-98dd-804ad3a43ae5 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:16.245646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:16.290949 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5b9cb-13da-455d-98dd-804ad3a43ae5 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:16.326062 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47800fdd-00f0-4ae4-a564-4be4067a2f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:16.326693 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47800fdd-00f0-4ae4-a564-4be4067a2f62 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:16.326974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47800fdd-00f0-4ae4-a564-4be4067a2f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:16.327246 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47800fdd-00f0-4ae4-a564-4be4067a2f62 None admin] PATH_INFO: `/v3/users/6cc5c705e0334edf9f1c824f9fa5962d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:16.340311 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47800fdd-00f0-4ae4-a564-4be4067a2f62 None admin] RBAC: Authorizing `identity:get_user(user_id=6cc5c705e0334edf9f1c824f9fa5962d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:16.341774 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47800fdd-00f0-4ae4-a564-4be4067a2f62 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:16.343371 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 423/847] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:53:16 2022] GET /identity/v3/users/6cc5c705e0334edf9f1c824f9fa5962d => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:16.349366 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c10b0d-f5d8-46d3-9637-d83b2e2bab03 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:16.352172 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:16.447743 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c10b0d-f5d8-46d3-9637-d83b2e2bab03 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:16.480623 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3ec463-d16b-495c-bdda-ecf90cc0176b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:16.481265 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ec463-d16b-495c-bdda-ecf90cc0176b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:16.481540 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ec463-d16b-495c-bdda-ecf90cc0176b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:16.481809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ec463-d16b-495c-bdda-ecf90cc0176b None admin] PATH_INFO: `/v3/projects/f768d5001c4d42a9ad55bb5dfa58ef62` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:16.482903 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3ec463-d16b-495c-bdda-ecf90cc0176b None admin] RBAC: Authorizing `identity:get_project(project_id=f768d5001c4d42a9ad55bb5dfa58ef62)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:16.484309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3ec463-d16b-495c-bdda-ecf90cc0176b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:16.485987 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 425/848] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:53:16 2022] GET /identity/v3/projects/f768d5001c4d42a9ad55bb5dfa58ef62 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:53:16.491909 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47800fdd-00f0-4ae4-a564-4be4067a2f62 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:16.588326 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47800fdd-00f0-4ae4-a564-4be4067a2f62 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:16.598424 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:16.621744 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf38da1-92ca-4560-8043-c2b20340ceac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:16.622386 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf38da1-92ca-4560-8043-c2b20340ceac None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:16.622666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf38da1-92ca-4560-8043-c2b20340ceac None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:16.622940 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf38da1-92ca-4560-8043-c2b20340ceac None admin] PATH_INFO: `/v3/projects/f768d5001c4d42a9ad55bb5dfa58ef62/users/6cc5c705e0334edf9f1c824f9fa5962d/roles/49dae0405c764c96b7bfbcf9d9efd011` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:16.625087 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf38da1-92ca-4560-8043-c2b20340ceac None admin] RBAC: Authorizing `identity:create_grant(project_id=f768d5001c4d42a9ad55bb5dfa58ef62, user_id=6cc5c705e0334edf9f1c824f9fa5962d, role_id=49dae0405c764c96b7bfbcf9d9efd011)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:16.627145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf38da1-92ca-4560-8043-c2b20340ceac None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:16.639460 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 424/849] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:53:16 2022] PUT /identity/v3/projects/f768d5001c4d42a9ad55bb5dfa58ef62/users/6cc5c705e0334edf9f1c824f9fa5962d/roles/49dae0405c764c96b7bfbcf9d9efd011 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:53:18.741909 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47daf311-476c-4464-a230-6c87d4576070 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:18.742461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47daf311-476c-4464-a230-6c87d4576070 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:18.742461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47daf311-476c-4464-a230-6c87d4576070 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:18.743751 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 426/850] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:53:18.748819 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777c3ac-1922-4656-aa2b-e071532f457c None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:18.749242 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777c3ac-1922-4656-aa2b-e071532f457c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:18.749355 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777c3ac-1922-4656-aa2b-e071532f457c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:18.768195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-e777c3ac-1922-4656-aa2b-e071532f457c None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:53:18.769410 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e777c3ac-1922-4656-aa2b-e071532f457c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:53:18.835073 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 425/851] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:53:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:53:19.108537 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47daf311-476c-4464-a230-6c87d4576070 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:19.179663 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:19.211294 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47daf311-476c-4464-a230-6c87d4576070 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:19.244536 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6764c57-7264-46c9-97fa-274783829385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:19.245177 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6764c57-7264-46c9-97fa-274783829385 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:19.245456 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6764c57-7264-46c9-97fa-274783829385 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:19.245728 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6764c57-7264-46c9-97fa-274783829385 None admin] PATH_INFO: `/v3/roles/49dae0405c764c96b7bfbcf9d9efd011` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:19.247093 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6764c57-7264-46c9-97fa-274783829385 None admin] RBAC: Authorizing `identity:get_role(role_id=49dae0405c764c96b7bfbcf9d9efd011)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:19.248449 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6764c57-7264-46c9-97fa-274783829385 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:19.249773 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 427/852] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:53:19 2022] GET /identity/v3/roles/49dae0405c764c96b7bfbcf9d9efd011 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:19.256162 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777c3ac-1922-4656-aa2b-e071532f457c None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:19.349226 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:19.352314 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777c3ac-1922-4656-aa2b-e071532f457c None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:19.386957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82413e-167b-4a95-873f-1f7df9aff930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:19.387582 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82413e-167b-4a95-873f-1f7df9aff930 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:19.387887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82413e-167b-4a95-873f-1f7df9aff930 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:19.388157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82413e-167b-4a95-873f-1f7df9aff930 None admin] PATH_INFO: `/v3/users/6cc5c705e0334edf9f1c824f9fa5962d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:19.389305 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c82413e-167b-4a95-873f-1f7df9aff930 None admin] RBAC: Authorizing `identity:get_user(user_id=6cc5c705e0334edf9f1c824f9fa5962d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:19.390727 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c82413e-167b-4a95-873f-1f7df9aff930 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:19.392331 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 426/853] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:53:19 2022] GET /identity/v3/users/6cc5c705e0334edf9f1c824f9fa5962d => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:19.398105 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6764c57-7264-46c9-97fa-274783829385 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:19.493830 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6764c57-7264-46c9-97fa-274783829385 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:19.527304 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a2d56-ffce-42c3-8977-81038e9cb2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:19.527895 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a2d56-ffce-42c3-8977-81038e9cb2f0 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:19.528173 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a2d56-ffce-42c3-8977-81038e9cb2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:19.528443 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a2d56-ffce-42c3-8977-81038e9cb2f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:19.536638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:19.594476 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45a2d56-ffce-42c3-8977-81038e9cb2f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:19.595977 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45a2d56-ffce-42c3-8977-81038e9cb2f0 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:19.691342 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 428/854] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:53:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 05:53:19.697020 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82413e-167b-4a95-873f-1f7df9aff930 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:19.793923 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82413e-167b-4a95-873f-1f7df9aff930 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:19.826912 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c5ebc-df2f-490b-a3a6-25a30e0c2937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:19.827520 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c5ebc-df2f-490b-a3a6-25a30e0c2937 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:19.827798 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c5ebc-df2f-490b-a3a6-25a30e0c2937 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:19.828066 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c5ebc-df2f-490b-a3a6-25a30e0c2937 None admin] PATH_INFO: `/v3/projects/f768d5001c4d42a9ad55bb5dfa58ef62` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:19.829450 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c5ebc-df2f-490b-a3a6-25a30e0c2937 None admin] RBAC: Authorizing `identity:get_project(project_id=f768d5001c4d42a9ad55bb5dfa58ef62)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:19.830856 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c5ebc-df2f-490b-a3a6-25a30e0c2937 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:19.832458 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 427/855] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:53:19 2022] GET /identity/v3/projects/f768d5001c4d42a9ad55bb5dfa58ef62 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:19.838601 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a2d56-ffce-42c3-8977-81038e9cb2f0 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:19.877004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:19.935483 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a2d56-ffce-42c3-8977-81038e9cb2f0 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:19.968144 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b1f4a-7ca7-478f-bda8-5cf31227c19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:19.968684 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b1f4a-7ca7-478f-bda8-5cf31227c19e None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:19.969564 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b1f4a-7ca7-478f-bda8-5cf31227c19e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:19.969834 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b1f4a-7ca7-478f-bda8-5cf31227c19e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:19.970545 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b1f4a-7ca7-478f-bda8-5cf31227c19e None admin] RBAC: Adding query filter params (role.id=49dae0405c764c96b7bfbcf9d9efd011, scope.project.id=f768d5001c4d42a9ad55bb5dfa58ef62, user.id=6cc5c705e0334edf9f1c824f9fa5962d) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:19.970853 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b1f4a-7ca7-478f-bda8-5cf31227c19e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:19.972271 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b1f4a-7ca7-478f-bda8-5cf31227c19e None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:19.977833 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 429/856] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:53:19 2022] GET /identity/v3/role_assignments?user.id=6cc5c705e0334edf9f1c824f9fa5962d&scope.project.id=f768d5001c4d42a9ad55bb5dfa58ef62&role.id=49dae0405c764c96b7bfbcf9d9efd011 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:22.107368 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb69c8-1d08-4307-9c7d-0fe5732e4840 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:22.107897 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb69c8-1d08-4307-9c7d-0fe5732e4840 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:22.107897 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb69c8-1d08-4307-9c7d-0fe5732e4840 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:22.109253 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 428/857] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:53:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:53:22.114516 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f32f5c-012b-4b05-bf8a-398c899cd00f None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:22.114938 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f32f5c-012b-4b05-bf8a-398c899cd00f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:22.115030 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f32f5c-012b-4b05-bf8a-398c899cd00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:22.133025 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-67f32f5c-012b-4b05-bf8a-398c899cd00f None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:53:22.134179 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-67f32f5c-012b-4b05-bf8a-398c899cd00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:53:22.198283 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 430/858] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:53:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 05:53:22.469916 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb69c8-1d08-4307-9c7d-0fe5732e4840 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:22.504873 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:22.568833 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb69c8-1d08-4307-9c7d-0fe5732e4840 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:22.601414 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec3b26-069d-4018-a231-988451691e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:22.602028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec3b26-069d-4018-a231-988451691e6f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:22.602303 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec3b26-069d-4018-a231-988451691e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:22.602567 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec3b26-069d-4018-a231-988451691e6f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:22.603676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec3b26-069d-4018-a231-988451691e6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:22.605117 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec3b26-069d-4018-a231-988451691e6f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:22.606647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 429/859] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:53:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:22.612342 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f32f5c-012b-4b05-bf8a-398c899cd00f None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:22.643920 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:22.709834 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f32f5c-012b-4b05-bf8a-398c899cd00f None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:22.741819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03934c-0363-4945-8ba5-b899bf767587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:22.742366 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03934c-0363-4945-8ba5-b899bf767587 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:22.742642 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03934c-0363-4945-8ba5-b899bf767587 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:22.742913 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03934c-0363-4945-8ba5-b899bf767587 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:22.743556 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03934c-0363-4945-8ba5-b899bf767587 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:22.744995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03934c-0363-4945-8ba5-b899bf767587 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:22.772044 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 431/860] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Oct 20 05:53:22 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:53:24.899857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9ed3e-64a7-402f-bc7f-59d66f96850c None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:24.900413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9ed3e-64a7-402f-bc7f-59d66f96850c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:24.900413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9ed3e-64a7-402f-bc7f-59d66f96850c None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:24.901757 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 430/861] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:53:24.907489 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1cf56a-659a-4b76-84b8-05fe1ff706d3 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:24.907865 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1cf56a-659a-4b76-84b8-05fe1ff706d3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:24.908025 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1cf56a-659a-4b76-84b8-05fe1ff706d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:24.926335 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-9b1cf56a-659a-4b76-84b8-05fe1ff706d3 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:53:24.927494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-9b1cf56a-659a-4b76-84b8-05fe1ff706d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:53:24.989795 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 432/862] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:53:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:53:25.255318 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9ed3e-64a7-402f-bc7f-59d66f96850c None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:25.354593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9ed3e-64a7-402f-bc7f-59d66f96850c None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:25.356850 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:25.388679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e7ff7-0b40-4a23-9e60-99c6ccbcee27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:25.389327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e7ff7-0b40-4a23-9e60-99c6ccbcee27 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:25.389607 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e7ff7-0b40-4a23-9e60-99c6ccbcee27 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:25.389881 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e7ff7-0b40-4a23-9e60-99c6ccbcee27 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:25.391154 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e7ff7-0b40-4a23-9e60-99c6ccbcee27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:25.392559 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e7ff7-0b40-4a23-9e60-99c6ccbcee27 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:25.394154 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 431/863] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:53:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:25.399879 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1cf56a-659a-4b76-84b8-05fe1ff706d3 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:25.414508 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:25.497252 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1cf56a-659a-4b76-84b8-05fe1ff706d3 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:25.642781 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38497984-850d-4938-add7-2c0a37d24129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:25.643381 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38497984-850d-4938-add7-2c0a37d24129 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:25.643659 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38497984-850d-4938-add7-2c0a37d24129 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:25.643928 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38497984-850d-4938-add7-2c0a37d24129 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:25.644566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38497984-850d-4938-add7-2c0a37d24129 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:25.645993 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38497984-850d-4938-add7-2c0a37d24129 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:25.666312 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 433/864] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Oct 20 05:53:25 2022] POST /identity/v3/users => generated 286 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:53:27.799319 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187aa35-5e69-4435-99c0-4cce16ea3fba None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:27.799850 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187aa35-5e69-4435-99c0-4cce16ea3fba None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:27.799850 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187aa35-5e69-4435-99c0-4cce16ea3fba None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:27.801234 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 432/865] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:53:27.806349 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da959233-7e82-45e8-890a-26b2a88374d3 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:27.806752 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da959233-7e82-45e8-890a-26b2a88374d3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:27.806924 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da959233-7e82-45e8-890a-26b2a88374d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:27.824484 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-da959233-7e82-45e8-890a-26b2a88374d3 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:53:27.825712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-da959233-7e82-45e8-890a-26b2a88374d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:53:27.885561 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:27.890889 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 434/866] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:53:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:53:28.162310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187aa35-5e69-4435-99c0-4cce16ea3fba None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:28.260378 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187aa35-5e69-4435-99c0-4cce16ea3fba None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:28.293596 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88c794-61dd-4678-8bfb-e4a691f4fb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:28.294215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88c794-61dd-4678-8bfb-e4a691f4fb7a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:28.294495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88c794-61dd-4678-8bfb-e4a691f4fb7a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:28.294769 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88c794-61dd-4678-8bfb-e4a691f4fb7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:28.306903 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd88c794-61dd-4678-8bfb-e4a691f4fb7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:53:28.311695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-dd88c794-61dd-4678-8bfb-e4a691f4fb7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:53:28.311695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:28.311695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:53:28.311695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:53:28.311695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:53:28.311695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:53:28.311695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:53:28.311695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:53:28.311695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:53:28.311695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:53:28.311695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:53:28.311695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:53:28.311695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:53:28.311695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:53:28.311695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:53:28.311695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:53:28.312974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:53:28.312974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:53:28.312974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:53:28.312974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:53:28.312974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:53:28.312974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:53:28.312974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:53:28.312974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:53:28.312974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:53:28.312974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:53:28.312974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:53:28.312974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:53:28.312974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:53:28.312974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:53:28.312974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:53:28.312974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:53:28.314059 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:53:28.314059 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:53:28.314059 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:53:28.314059 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:53:28.314059 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:53:28.314059 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:53:28.314059 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:53:28.314059 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:53:28.314059 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:53:28.314059 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:53:28.314059 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:53:28.314059 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:53:28.314059 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 433/867] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Oct 20 05:53:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:53:28.318921 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da959233-7e82-45e8-890a-26b2a88374d3 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:28.412967 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da959233-7e82-45e8-890a-26b2a88374d3 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:28.446020 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e908eb-f539-43a1-820c-3d633a85a38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:28.446611 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e908eb-f539-43a1-820c-3d633a85a38b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:28.446883 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e908eb-f539-43a1-820c-3d633a85a38b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:28.447150 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e908eb-f539-43a1-820c-3d633a85a38b None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:28.447789 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e908eb-f539-43a1-820c-3d633a85a38b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:28.448080 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e908eb-f539-43a1-820c-3d633a85a38b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:28.449447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e908eb-f539-43a1-820c-3d633a85a38b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:28.457609 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 435/868] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:53:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:28.463439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88c794-61dd-4678-8bfb-e4a691f4fb7a None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:28.495308 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:28.562055 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88c794-61dd-4678-8bfb-e4a691f4fb7a None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:28.595079 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8317a05a-0d2d-42ef-8ea4-bb460a290cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:28.595690 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317a05a-0d2d-42ef-8ea4-bb460a290cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:28.595965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317a05a-0d2d-42ef-8ea4-bb460a290cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:28.596235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317a05a-0d2d-42ef-8ea4-bb460a290cb6 None admin] PATH_INFO: `/v3/users/08502e6ef1694af18aac959d463b6e93` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:28.608543 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8317a05a-0d2d-42ef-8ea4-bb460a290cb6 None admin] RBAC: Authorizing `identity:get_user(user_id=08502e6ef1694af18aac959d463b6e93)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:28.610050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8317a05a-0d2d-42ef-8ea4-bb460a290cb6 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:28.611603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 434/869] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:53:28 2022] GET /identity/v3/users/08502e6ef1694af18aac959d463b6e93 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:28.617366 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e908eb-f539-43a1-820c-3d633a85a38b None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:28.677529 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:28.715406 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e908eb-f539-43a1-820c-3d633a85a38b None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:28.747132 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f60ded-db98-45b0-9dd6-511aa0a7a23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:28.747746 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f60ded-db98-45b0-9dd6-511aa0a7a23a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:28.748022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f60ded-db98-45b0-9dd6-511aa0a7a23a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:28.748291 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f60ded-db98-45b0-9dd6-511aa0a7a23a None admin] PATH_INFO: `/v3/projects/9755035832684e7eb9e7a035a3d45194` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:28.749393 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f60ded-db98-45b0-9dd6-511aa0a7a23a None admin] RBAC: Authorizing `identity:get_project(project_id=9755035832684e7eb9e7a035a3d45194)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:28.751541 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f60ded-db98-45b0-9dd6-511aa0a7a23a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:28.753150 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 436/870] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:53:28 2022] GET /identity/v3/projects/9755035832684e7eb9e7a035a3d45194 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:28.758979 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8317a05a-0d2d-42ef-8ea4-bb460a290cb6 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:28.844830 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:28.854071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8317a05a-0d2d-42ef-8ea4-bb460a290cb6 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:28.886373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fa3de-b022-4702-b8e5-2d02049211da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:28.887033 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fa3de-b022-4702-b8e5-2d02049211da None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:28.887305 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fa3de-b022-4702-b8e5-2d02049211da None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:28.887572 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fa3de-b022-4702-b8e5-2d02049211da None admin] PATH_INFO: `/v3/projects/9755035832684e7eb9e7a035a3d45194/users/08502e6ef1694af18aac959d463b6e93/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:28.889444 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fa3de-b022-4702-b8e5-2d02049211da None admin] RBAC: Authorizing `identity:create_grant(project_id=9755035832684e7eb9e7a035a3d45194, user_id=08502e6ef1694af18aac959d463b6e93, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:28.891412 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fa3de-b022-4702-b8e5-2d02049211da None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:28.904201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 435/871] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:53:28 2022] PUT /identity/v3/projects/9755035832684e7eb9e7a035a3d45194/users/08502e6ef1694af18aac959d463b6e93/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:53:31.011206 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b9e39-bb16-413b-bc02-8c99795bfe97 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:31.011754 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b9e39-bb16-413b-bc02-8c99795bfe97 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:31.011754 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b9e39-bb16-413b-bc02-8c99795bfe97 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:31.013089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 437/872] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:53:31.018561 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827c0f7-8a21-426a-9398-559279f8622e None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:31.018987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827c0f7-8a21-426a-9398-559279f8622e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:31.019106 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827c0f7-8a21-426a-9398-559279f8622e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:31.037449 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d827c0f7-8a21-426a-9398-559279f8622e None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:53:31.038621 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d827c0f7-8a21-426a-9398-559279f8622e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:53:31.102540 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 436/873] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:53:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:53:31.368703 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b9e39-bb16-413b-bc02-8c99795bfe97 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:31.466989 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b9e39-bb16-413b-bc02-8c99795bfe97 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:31.494012 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:31.500774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3d1c6-9880-4a07-ab04-4a8e21c7d634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:31.501392 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3d1c6-9880-4a07-ab04-4a8e21c7d634 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:31.501666 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3d1c6-9880-4a07-ab04-4a8e21c7d634 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:31.501934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3d1c6-9880-4a07-ab04-4a8e21c7d634 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:31.511873 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3d1c6-9880-4a07-ab04-4a8e21c7d634 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:53:31.516552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-87a3d1c6-9880-4a07-ab04-4a8e21c7d634 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:53:31.516552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:31.516552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:53:31.516552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:53:31.516552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:53:31.516552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:53:31.516552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:53:31.516552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:53:31.516552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:53:31.516552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:53:31.516552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:53:31.516552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:53:31.516552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:53:31.516552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:53:31.516552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:53:31.516552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:53:31.517714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:53:31.517714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:53:31.517714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:53:31.517714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:53:31.517714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:53:31.517714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:53:31.517714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:53:31.517714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:53:31.517714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:53:31.517714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:53:31.517714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:53:31.517714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:53:31.517714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:53:31.517714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:53:31.517714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:53:31.517714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:53:31.518734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:53:31.518734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:53:31.518734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:53:31.518734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:53:31.518734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:53:31.518734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:53:31.518734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:53:31.518734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:53:31.518734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:53:31.518734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:53:31.518734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:53:31.518734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:53:31.518734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 438/874] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Oct 20 05:53:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:53:31.523697 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827c0f7-8a21-426a-9398-559279f8622e None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:31.599388 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:31.618659 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827c0f7-8a21-426a-9398-559279f8622e None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:31.650934 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa8c08-9bca-414e-9474-ceeec29ab443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:31.651531 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa8c08-9bca-414e-9474-ceeec29ab443 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:31.651808 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa8c08-9bca-414e-9474-ceeec29ab443 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:31.652076 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa8c08-9bca-414e-9474-ceeec29ab443 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:31.652745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa8c08-9bca-414e-9474-ceeec29ab443 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:31.653048 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa8c08-9bca-414e-9474-ceeec29ab443 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:31.654404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa8c08-9bca-414e-9474-ceeec29ab443 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:31.662899 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 437/875] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:53:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:31.669248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3d1c6-9880-4a07-ab04-4a8e21c7d634 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:31.764398 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3d1c6-9880-4a07-ab04-4a8e21c7d634 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:31.797335 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf54f90-0b33-4d24-ba6e-06b41c57c7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:31.797947 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf54f90-0b33-4d24-ba6e-06b41c57c7cc None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:31.798227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf54f90-0b33-4d24-ba6e-06b41c57c7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:31.798499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf54f90-0b33-4d24-ba6e-06b41c57c7cc None admin] PATH_INFO: `/v3/users/08502e6ef1694af18aac959d463b6e93` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:31.799608 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf54f90-0b33-4d24-ba6e-06b41c57c7cc None admin] RBAC: Authorizing `identity:get_user(user_id=08502e6ef1694af18aac959d463b6e93)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:31.801042 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf54f90-0b33-4d24-ba6e-06b41c57c7cc None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:31.802570 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 439/876] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:53:31 2022] GET /identity/v3/users/08502e6ef1694af18aac959d463b6e93 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:31.808217 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa8c08-9bca-414e-9474-ceeec29ab443 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:31.904550 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa8c08-9bca-414e-9474-ceeec29ab443 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:31.936625 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2999ae5c-ac48-4524-8d37-09ee06fe8401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:31.937320 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999ae5c-ac48-4524-8d37-09ee06fe8401 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:31.937659 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999ae5c-ac48-4524-8d37-09ee06fe8401 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:31.937892 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999ae5c-ac48-4524-8d37-09ee06fe8401 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:31.949698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:32.003706 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2999ae5c-ac48-4524-8d37-09ee06fe8401 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:32.005169 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2999ae5c-ac48-4524-8d37-09ee06fe8401 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:32.099165 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 438/877] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:53:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 05:53:32.105029 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf54f90-0b33-4d24-ba6e-06b41c57c7cc None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:32.161707 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:32.200534 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf54f90-0b33-4d24-ba6e-06b41c57c7cc None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:32.233563 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805f90c-b647-4b95-ab52-da41a5adfa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:32.234289 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805f90c-b647-4b95-ab52-da41a5adfa3f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:32.234571 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805f90c-b647-4b95-ab52-da41a5adfa3f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:32.234837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805f90c-b647-4b95-ab52-da41a5adfa3f None admin] PATH_INFO: `/v3/projects/9755035832684e7eb9e7a035a3d45194` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:32.235940 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0805f90c-b647-4b95-ab52-da41a5adfa3f None admin] RBAC: Authorizing `identity:get_project(project_id=9755035832684e7eb9e7a035a3d45194)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:32.237391 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0805f90c-b647-4b95-ab52-da41a5adfa3f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:32.238966 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 440/878] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:53:32 2022] GET /identity/v3/projects/9755035832684e7eb9e7a035a3d45194 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:32.245018 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2999ae5c-ac48-4524-8d37-09ee06fe8401 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:32.286595 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:32.343392 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2999ae5c-ac48-4524-8d37-09ee06fe8401 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:32.376029 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b46ee-1509-4c7c-b621-78b441728f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:32.376618 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b46ee-1509-4c7c-b621-78b441728f3a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:32.376931 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b46ee-1509-4c7c-b621-78b441728f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:32.377207 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b46ee-1509-4c7c-b621-78b441728f3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:32.377918 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b46ee-1509-4c7c-b621-78b441728f3a None admin] RBAC: Adding query filter params (role.id=d8debbb193834fac9cc25f81726f3a67, scope.project.id=9755035832684e7eb9e7a035a3d45194, user.id=08502e6ef1694af18aac959d463b6e93) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:32.378224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b46ee-1509-4c7c-b621-78b441728f3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:32.379631 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b46ee-1509-4c7c-b621-78b441728f3a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:32.385457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 439/879] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:53:32 2022] GET /identity/v3/role_assignments?user.id=08502e6ef1694af18aac959d463b6e93&scope.project.id=9755035832684e7eb9e7a035a3d45194&role.id=d8debbb193834fac9cc25f81726f3a67 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:34.489239 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5c864-843a-4714-9826-b0a527901117 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:34.489790 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5c864-843a-4714-9826-b0a527901117 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:34.489790 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5c864-843a-4714-9826-b0a527901117 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:34.491060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 441/880] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:53:34.496284 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ea6bf-c052-4a65-af26-404ad04092fb None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:34.496669 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ea6bf-c052-4a65-af26-404ad04092fb None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:34.496866 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ea6bf-c052-4a65-af26-404ad04092fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:34.515393 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-443ea6bf-c052-4a65-af26-404ad04092fb None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:53:34.516638 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-443ea6bf-c052-4a65-af26-404ad04092fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:53:34.580660 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 440/881] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:53:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:53:34.847210 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5c864-843a-4714-9826-b0a527901117 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:34.945034 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5c864-843a-4714-9826-b0a527901117 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:34.969482 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:34.978154 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ddee2-f1fe-415e-afec-c5182673beac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:34.978752 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ddee2-f1fe-415e-afec-c5182673beac None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:34.979030 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ddee2-f1fe-415e-afec-c5182673beac None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:34.979301 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ddee2-f1fe-415e-afec-c5182673beac None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:34.979947 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246ddee2-f1fe-415e-afec-c5182673beac None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:34.981324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246ddee2-f1fe-415e-afec-c5182673beac None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:34.996463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 442/882] 10.222.0.20 () {60 vars in 1138 bytes} [Thu Oct 20 05:53:34 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:53:37.144386 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95853885-9c14-432d-996a-84c68adb27cb None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:37.144977 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95853885-9c14-432d-996a-84c68adb27cb None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:37.144977 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95853885-9c14-432d-996a-84c68adb27cb None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:37.146294 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 441/883] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:53:37.151582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6e12e-1852-4cef-ba9b-bbc7e4aaf9bc None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:37.151998 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6e12e-1852-4cef-ba9b-bbc7e4aaf9bc None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:37.152107 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6e12e-1852-4cef-ba9b-bbc7e4aaf9bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:37.169620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-2af6e12e-1852-4cef-ba9b-bbc7e4aaf9bc None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:53:37.170777 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2af6e12e-1852-4cef-ba9b-bbc7e4aaf9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:53:37.234336 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 443/884] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:53:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:53:37.508366 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95853885-9c14-432d-996a-84c68adb27cb None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:37.544637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:37.608249 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95853885-9c14-432d-996a-84c68adb27cb None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:37.642142 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df9690-fae9-4ca0-a346-8907e75aed6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:37.642757 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df9690-fae9-4ca0-a346-8907e75aed6a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:37.643032 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df9690-fae9-4ca0-a346-8907e75aed6a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:37.643302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df9690-fae9-4ca0-a346-8907e75aed6a None admin] PATH_INFO: `/v3/domains/cb2e04d4c1b14035b81e0820d903ebab` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:37.653210 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df9690-fae9-4ca0-a346-8907e75aed6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=cb2e04d4c1b14035b81e0820d903ebab)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:37.654630 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df9690-fae9-4ca0-a346-8907e75aed6a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:37.657265 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 442/885] 10.222.0.20 () {56 vars in 1183 bytes} [Thu Oct 20 05:53:37 2022] GET /identity/v3/domains/cb2e04d4c1b14035b81e0820d903ebab => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:37.663125 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6e12e-1852-4cef-ba9b-bbc7e4aaf9bc None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:37.756770 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6e12e-1852-4cef-ba9b-bbc7e4aaf9bc None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:37.773093 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:37.791391 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81435bdb-a67c-42a1-b32d-46e78c30615a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:37.791985 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81435bdb-a67c-42a1-b32d-46e78c30615a None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:37.792259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81435bdb-a67c-42a1-b32d-46e78c30615a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:37.792528 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81435bdb-a67c-42a1-b32d-46e78c30615a None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:37.793195 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81435bdb-a67c-42a1-b32d-46e78c30615a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:37.794609 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81435bdb-a67c-42a1-b32d-46e78c30615a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:37.820605 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 444/886] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Oct 20 05:53:37 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 05:53:39.961861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676ea07-bf81-476f-a52c-688d644a8eef None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:39.962397 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676ea07-bf81-476f-a52c-688d644a8eef None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:39.962397 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676ea07-bf81-476f-a52c-688d644a8eef None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:39.963718 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 443/887] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:53:39.968958 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd4f85-98bb-40e1-88c2-0765f7e6107d None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:39.969376 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd4f85-98bb-40e1-88c2-0765f7e6107d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:39.969502 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd4f85-98bb-40e1-88c2-0765f7e6107d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:39.986973 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-cdfd4f85-98bb-40e1-88c2-0765f7e6107d None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:53:39.988127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-cdfd4f85-98bb-40e1-88c2-0765f7e6107d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:53:40.050844 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 445/888] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:53:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:53:40.317972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676ea07-bf81-476f-a52c-688d644a8eef None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:40.408658 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:40.415273 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676ea07-bf81-476f-a52c-688d644a8eef None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:40.448048 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3fb75-8678-41bb-98d3-60d4d9883918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:40.448656 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3fb75-8678-41bb-98d3-60d4d9883918 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:40.448957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3fb75-8678-41bb-98d3-60d4d9883918 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:40.449226 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3fb75-8678-41bb-98d3-60d4d9883918 None admin] PATH_INFO: `/v3/domains/cb2e04d4c1b14035b81e0820d903ebab` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:40.450309 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3fb75-8678-41bb-98d3-60d4d9883918 None admin] RBAC: Authorizing `identity:get_domain(domain_id=cb2e04d4c1b14035b81e0820d903ebab)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:40.451746 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3fb75-8678-41bb-98d3-60d4d9883918 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:40.453526 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 444/889] 10.222.0.20 () {56 vars in 1183 bytes} [Thu Oct 20 05:53:40 2022] GET /identity/v3/domains/cb2e04d4c1b14035b81e0820d903ebab => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:40.459160 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd4f85-98bb-40e1-88c2-0765f7e6107d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:40.552091 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:40.556256 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd4f85-98bb-40e1-88c2-0765f7e6107d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:40.589734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1865c4-7ce1-4b69-bda4-6baa0624b595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:40.590327 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1865c4-7ce1-4b69-bda4-6baa0624b595 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:40.590599 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1865c4-7ce1-4b69-bda4-6baa0624b595 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:40.590868 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1865c4-7ce1-4b69-bda4-6baa0624b595 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:40.591501 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1865c4-7ce1-4b69-bda4-6baa0624b595 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:40.593481 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1865c4-7ce1-4b69-bda4-6baa0624b595 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:40.614174 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 446/890] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Oct 20 05:53:40 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:53:42.801481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d2eb5-30f7-4de1-aa79-4ce8225a6d1f None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:42.802043 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d2eb5-30f7-4de1-aa79-4ce8225a6d1f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:42.802043 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d2eb5-30f7-4de1-aa79-4ce8225a6d1f None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:42.803372 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 445/891] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:53:42.808559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706e62b-968d-4dc7-bdc8-8213df7bd3e6 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:42.808990 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706e62b-968d-4dc7-bdc8-8213df7bd3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:42.809166 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706e62b-968d-4dc7-bdc8-8213df7bd3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:42.827930 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-9706e62b-968d-4dc7-bdc8-8213df7bd3e6 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:53:42.829539 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-9706e62b-968d-4dc7-bdc8-8213df7bd3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:53:42.893924 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 447/892] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:53:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 05:53:43.161130 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d2eb5-30f7-4de1-aa79-4ce8225a6d1f None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:43.258928 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d2eb5-30f7-4de1-aa79-4ce8225a6d1f None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:43.290545 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b91cf-0b88-41e2-a2f7-743e866a149c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:43.291154 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b91cf-0b88-41e2-a2f7-743e866a149c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:43.291426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b91cf-0b88-41e2-a2f7-743e866a149c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:43.291689 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b91cf-0b88-41e2-a2f7-743e866a149c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:43.302521 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b91cf-0b88-41e2-a2f7-743e866a149c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:53:43.307267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-bf4b91cf-0b88-41e2-a2f7-743e866a149c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:53:43.307267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:43.307267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:53:43.307267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:53:43.307267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:53:43.307267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:53:43.307267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:53:43.307267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:53:43.307267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:53:43.307267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:53:43.307267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:53:43.307267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:53:43.307267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:53:43.307267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:53:43.307267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:53:43.307267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:53:43.308370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:53:43.308370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:53:43.308370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:53:43.308370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:53:43.308370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:53:43.308370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:53:43.308370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:53:43.308370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:53:43.308370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:53:43.308370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:53:43.308370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:53:43.308370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:53:43.308370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:53:43.308370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:53:43.308370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:53:43.308370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:53:43.309432 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:53:43.309432 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:53:43.309432 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:53:43.309432 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:53:43.309432 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:53:43.309432 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:53:43.309432 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:53:43.309432 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:53:43.309432 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:53:43.309432 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:53:43.309432 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:53:43.309432 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:53:43.309432 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 446/893] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Oct 20 05:53:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:53:43.315023 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706e62b-968d-4dc7-bdc8-8213df7bd3e6 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:43.393784 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:43.409435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706e62b-968d-4dc7-bdc8-8213df7bd3e6 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:43.441695 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c340e-242a-44df-8fc7-bdaf7af6bed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:43.442233 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c340e-242a-44df-8fc7-bdaf7af6bed9 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:43.442507 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c340e-242a-44df-8fc7-bdaf7af6bed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:43.442772 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c340e-242a-44df-8fc7-bdaf7af6bed9 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:43.443419 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c340e-242a-44df-8fc7-bdaf7af6bed9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:43.443713 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c340e-242a-44df-8fc7-bdaf7af6bed9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:43.445088 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c340e-242a-44df-8fc7-bdaf7af6bed9 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:43.453219 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 448/894] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:53:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:43.458940 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b91cf-0b88-41e2-a2f7-743e866a149c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:43.484492 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:43.555568 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b91cf-0b88-41e2-a2f7-743e866a149c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:43.588279 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bd4b1-08c9-4382-8279-a3a978d61001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:43.588914 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bd4b1-08c9-4382-8279-a3a978d61001 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:43.589190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bd4b1-08c9-4382-8279-a3a978d61001 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:43.589458 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bd4b1-08c9-4382-8279-a3a978d61001 None admin] PATH_INFO: `/v3/users/302c97669ea64bfdb71c34ec54d45122` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:43.601937 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315bd4b1-08c9-4382-8279-a3a978d61001 None admin] RBAC: Authorizing `identity:get_user(user_id=302c97669ea64bfdb71c34ec54d45122)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:43.603346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315bd4b1-08c9-4382-8279-a3a978d61001 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:43.604921 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 447/895] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:53:43 2022] GET /identity/v3/users/302c97669ea64bfdb71c34ec54d45122 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:43.610645 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c340e-242a-44df-8fc7-bdaf7af6bed9 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:43.705977 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c340e-242a-44df-8fc7-bdaf7af6bed9 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:43.737415 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10550ac-435b-44d7-a3c3-1412c167aaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:43.738027 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10550ac-435b-44d7-a3c3-1412c167aaaf None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:43.738300 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10550ac-435b-44d7-a3c3-1412c167aaaf None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:43.738565 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10550ac-435b-44d7-a3c3-1412c167aaaf None admin] PATH_INFO: `/v3/projects/cea315529df4493ca5907f9601bcc600` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:43.739639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10550ac-435b-44d7-a3c3-1412c167aaaf None admin] RBAC: Authorizing `identity:get_project(project_id=cea315529df4493ca5907f9601bcc600)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:43.741061 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10550ac-435b-44d7-a3c3-1412c167aaaf None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:43.742620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 449/896] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:53:43 2022] GET /identity/v3/projects/cea315529df4493ca5907f9601bcc600 => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:43.748609 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bd4b1-08c9-4382-8279-a3a978d61001 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:43.826616 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:43.845072 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bd4b1-08c9-4382-8279-a3a978d61001 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:43.877403 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7428dbff-c69f-4297-a7fb-0d793193c970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:43.878042 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7428dbff-c69f-4297-a7fb-0d793193c970 None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:43.878319 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7428dbff-c69f-4297-a7fb-0d793193c970 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:43.878589 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7428dbff-c69f-4297-a7fb-0d793193c970 None admin] PATH_INFO: `/v3/projects/cea315529df4493ca5907f9601bcc600/users/302c97669ea64bfdb71c34ec54d45122/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:43.880426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7428dbff-c69f-4297-a7fb-0d793193c970 None admin] RBAC: Authorizing `identity:create_grant(project_id=cea315529df4493ca5907f9601bcc600, user_id=302c97669ea64bfdb71c34ec54d45122, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:43.881859 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7428dbff-c69f-4297-a7fb-0d793193c970 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:43.894481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 448/897] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:53:43 2022] PUT /identity/v3/projects/cea315529df4493ca5907f9601bcc600/users/302c97669ea64bfdb71c34ec54d45122/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:53:46.034265 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8af0d2-1be0-45c5-b6bd-b5334af27c86 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:46.034797 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8af0d2-1be0-45c5-b6bd-b5334af27c86 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:46.034797 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8af0d2-1be0-45c5-b6bd-b5334af27c86 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:46.036127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 450/898] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:53:46.041277 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d258b2-9b05-4147-bcf4-f93e87ff70de None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:46.041702 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d258b2-9b05-4147-bcf4-f93e87ff70de None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:46.041817 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d258b2-9b05-4147-bcf4-f93e87ff70de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:46.060221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-f3d258b2-9b05-4147-bcf4-f93e87ff70de None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:53:46.061416 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f3d258b2-9b05-4147-bcf4-f93e87ff70de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:53:46.126050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 449/899] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:53:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 05:53:46.394649 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8af0d2-1be0-45c5-b6bd-b5334af27c86 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:46.411151 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:46.494858 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8af0d2-1be0-45c5-b6bd-b5334af27c86 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:46.529797 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4c479-f23b-4e06-9efb-ac67e86d9ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:46.531031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4c479-f23b-4e06-9efb-ac67e86d9ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:46.531409 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4c479-f23b-4e06-9efb-ac67e86d9ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:46.531688 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4c479-f23b-4e06-9efb-ac67e86d9ac7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:46.542000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa4c479-f23b-4e06-9efb-ac67e86d9ac7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:53:46.547055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-aaa4c479-f23b-4e06-9efb-ac67e86d9ac7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:53:46.547055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:53:46.547055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:53:46.547055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:53:46.547055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:53:46.547055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:53:46.547055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:53:46.547055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:53:46.547055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:53:46.547055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:53:46.547055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:53:46.547055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:53:46.547055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:53:46.547055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:53:46.547055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:53:46.547055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:53:46.548104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:53:46.548104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:53:46.548104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:53:46.548104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:53:46.548104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:53:46.548104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:53:46.548104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:53:46.548104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:53:46.548104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:53:46.548104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:53:46.548104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:53:46.548104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:53:46.548104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:53:46.548104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:53:46.548104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:53:46.548104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:53:46.549126 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:53:46.549126 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:53:46.549126 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:53:46.549126 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:53:46.549126 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:53:46.549126 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:53:46.549126 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:53:46.549126 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:53:46.549126 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:53:46.549126 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:53:46.549126 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:53:46.549126 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:53:46.549126 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 451/900] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Oct 20 05:53:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:53:46.554426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d258b2-9b05-4147-bcf4-f93e87ff70de None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:46.620831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:46.651238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d258b2-9b05-4147-bcf4-f93e87ff70de None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:46.685086 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52931d9e-2301-48a6-a336-cdfe53c14ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:46.685679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52931d9e-2301-48a6-a336-cdfe53c14ddb None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:46.685957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52931d9e-2301-48a6-a336-cdfe53c14ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:46.686224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52931d9e-2301-48a6-a336-cdfe53c14ddb None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:46.686991 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52931d9e-2301-48a6-a336-cdfe53c14ddb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:46.687288 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52931d9e-2301-48a6-a336-cdfe53c14ddb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:46.688640 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52931d9e-2301-48a6-a336-cdfe53c14ddb None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:46.696383 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 450/901] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:53:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:46.702381 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4c479-f23b-4e06-9efb-ac67e86d9ac7 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:46.774235 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:46.799344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4c479-f23b-4e06-9efb-ac67e86d9ac7 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:46.831500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14daf15a-5e82-4011-aab6-e3fb03f7045d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:46.832055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14daf15a-5e82-4011-aab6-e3fb03f7045d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:46.832332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14daf15a-5e82-4011-aab6-e3fb03f7045d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:46.832599 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14daf15a-5e82-4011-aab6-e3fb03f7045d None admin] PATH_INFO: `/v3/users/302c97669ea64bfdb71c34ec54d45122` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:46.833715 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14daf15a-5e82-4011-aab6-e3fb03f7045d None admin] RBAC: Authorizing `identity:get_user(user_id=302c97669ea64bfdb71c34ec54d45122)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:46.835125 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14daf15a-5e82-4011-aab6-e3fb03f7045d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:46.836659 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 452/902] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Oct 20 05:53:46 2022] GET /identity/v3/users/302c97669ea64bfdb71c34ec54d45122 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:46.842393 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52931d9e-2301-48a6-a336-cdfe53c14ddb None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:46.938461 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52931d9e-2301-48a6-a336-cdfe53c14ddb None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:46.971137 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb50203-e01e-426c-9910-ab090e3a5848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:46.971726 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb50203-e01e-426c-9910-ab090e3a5848 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:46.972005 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb50203-e01e-426c-9910-ab090e3a5848 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:46.972274 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb50203-e01e-426c-9910-ab090e3a5848 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:46.974735 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:47.037089 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb50203-e01e-426c-9910-ab090e3a5848 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:47.038514 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb50203-e01e-426c-9910-ab090e3a5848 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:47.132445 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 451/903] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:53:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 05:53:47.138458 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14daf15a-5e82-4011-aab6-e3fb03f7045d None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:47.237089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14daf15a-5e82-4011-aab6-e3fb03f7045d None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:47.269130 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33fabf4-460b-46c3-8da0-dd2a95424ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:47.269742 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33fabf4-460b-46c3-8da0-dd2a95424ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:47.270019 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33fabf4-460b-46c3-8da0-dd2a95424ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:47.270287 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33fabf4-460b-46c3-8da0-dd2a95424ee4 None admin] PATH_INFO: `/v3/projects/cea315529df4493ca5907f9601bcc600` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:47.271383 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33fabf4-460b-46c3-8da0-dd2a95424ee4 None admin] RBAC: Authorizing `identity:get_project(project_id=cea315529df4493ca5907f9601bcc600)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:47.272819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33fabf4-460b-46c3-8da0-dd2a95424ee4 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:47.274393 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 453/904] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Oct 20 05:53:47 2022] GET /identity/v3/projects/cea315529df4493ca5907f9601bcc600 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:47.280957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb50203-e01e-426c-9910-ab090e3a5848 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:53:47.313269 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:47.377366 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb50203-e01e-426c-9910-ab090e3a5848 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:53:47.410541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c1a72-546c-4a52-8100-99e8bd6db44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:53:47.411075 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c1a72-546c-4a52-8100-99e8bd6db44f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:53:47.411349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c1a72-546c-4a52-8100-99e8bd6db44f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:53:47.411616 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c1a72-546c-4a52-8100-99e8bd6db44f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:53:47.412316 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c1a72-546c-4a52-8100-99e8bd6db44f None admin] RBAC: Adding query filter params (role.id=d8debbb193834fac9cc25f81726f3a67, scope.project.id=cea315529df4493ca5907f9601bcc600, user.id=302c97669ea64bfdb71c34ec54d45122) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:53:47.412618 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c1a72-546c-4a52-8100-99e8bd6db44f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:53:47.414042 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c1a72-546c-4a52-8100-99e8bd6db44f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:53:47.419533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 452/905] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:53:47 2022] GET /identity/v3/role_assignments?user.id=302c97669ea64bfdb71c34ec54d45122&scope.project.id=cea315529df4493ca5907f9601bcc600&role.id=d8debbb193834fac9cc25f81726f3a67 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:53:57.264595 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=95 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:53:57.417034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=51 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:43.120132 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4101fa-16d6-4547-9e16-8fbba5b02b94 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:43.120132 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4101fa-16d6-4547-9e16-8fbba5b02b94 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:43.120132 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4101fa-16d6-4547-9e16-8fbba5b02b94 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:43.120132 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 454/906] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:54:43.127627 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06ddce-b0c2-425d-a1d8-494faffb84f1 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:43.128614 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06ddce-b0c2-425d-a1d8-494faffb84f1 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:43.129174 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06ddce-b0c2-425d-a1d8-494faffb84f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:43.151765 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-2c06ddce-b0c2-425d-a1d8-494faffb84f1 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:54:43.153645 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2c06ddce-b0c2-425d-a1d8-494faffb84f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:54:43.240095 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 453/907] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:54:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:54:43.585848 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4101fa-16d6-4547-9e16-8fbba5b02b94 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:43.720994 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4101fa-16d6-4547-9e16-8fbba5b02b94 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:43.765316 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83dbeeb-09b9-457f-b4d3-646f8b4d88b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:43.765988 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83dbeeb-09b9-457f-b4d3-646f8b4d88b7 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:43.766263 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83dbeeb-09b9-457f-b4d3-646f8b4d88b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:43.766545 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83dbeeb-09b9-457f-b4d3-646f8b4d88b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:43.780277 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83dbeeb-09b9-457f-b4d3-646f8b4d88b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:43.781978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83dbeeb-09b9-457f-b4d3-646f8b4d88b7 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:43.800983 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-d83dbeeb-09b9-457f-b4d3-646f8b4d88b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:54:43.800983 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:54:43.800983 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:54:43.800983 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:54:43.800983 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:54:43.800983 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:54:43.800983 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:54:43.800983 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:54:43.800983 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:54:43.800983 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:54:43.800983 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:54:43.800983 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:54:43.800983 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:54:43.800983 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:54:43.800983 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:54:43.800983 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:54:43.802164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:54:43.802164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:54:43.802164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:54:43.802164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:54:43.802164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:54:43.802164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:54:43.802164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:54:43.802164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:54:43.802164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:54:43.802164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:54:43.802164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:54:43.802164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:54:43.802164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:54:43.802164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:54:43.802164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:54:43.802164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:54:43.802164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:54:43.803152 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:54:43.803152 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:54:43.803152 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:54:43.803152 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 455/908] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:54:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:54:43.812080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06ddce-b0c2-425d-a1d8-494faffb84f1 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:43.970618 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06ddce-b0c2-425d-a1d8-494faffb84f1 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:44.010707 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:44.019120 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b0ca2-7ca4-45eb-93ee-6fe5efc4e7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:44.019843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b0ca2-7ca4-45eb-93ee-6fe5efc4e7d7 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:44.020247 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b0ca2-7ca4-45eb-93ee-6fe5efc4e7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:44.020498 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b0ca2-7ca4-45eb-93ee-6fe5efc4e7d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:44.021305 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b0ca2-7ca4-45eb-93ee-6fe5efc4e7d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:54:44.021627 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b0ca2-7ca4-45eb-93ee-6fe5efc4e7d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:44.023427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b0ca2-7ca4-45eb-93ee-6fe5efc4e7d7 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:44.037842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 454/909] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:54:44.044379 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83dbeeb-09b9-457f-b4d3-646f8b4d88b7 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:44.112657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:44.170082 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83dbeeb-09b9-457f-b4d3-646f8b4d88b7 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:44.217755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a21c17-49b3-4677-9b1c-6019fce6a035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:44.218684 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a21c17-49b3-4677-9b1c-6019fce6a035 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:44.219123 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a21c17-49b3-4677-9b1c-6019fce6a035 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:44.220487 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a21c17-49b3-4677-9b1c-6019fce6a035 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:44.221269 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a21c17-49b3-4677-9b1c-6019fce6a035 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:44.222696 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a21c17-49b3-4677-9b1c-6019fce6a035 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:44.252729 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 456/910] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Oct 20 05:54:44 2022] POST /identity/v3/users => generated 251 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:54:46.613401 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf7b54-faae-495f-b16a-ea747ff3f758 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:46.613999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf7b54-faae-495f-b16a-ea747ff3f758 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:46.613999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf7b54-faae-495f-b16a-ea747ff3f758 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:46.615534 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 455/911] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:54:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:54:46.621808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733a7d9-0ddd-4026-ad11-69a89ad67ad4 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:46.622258 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733a7d9-0ddd-4026-ad11-69a89ad67ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:46.622419 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733a7d9-0ddd-4026-ad11-69a89ad67ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:46.642098 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-5733a7d9-0ddd-4026-ad11-69a89ad67ad4 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:54:46.645726 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5733a7d9-0ddd-4026-ad11-69a89ad67ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:54:46.720349 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 457/912] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:54:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:54:47.019867 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf7b54-faae-495f-b16a-ea747ff3f758 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:47.145620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf7b54-faae-495f-b16a-ea747ff3f758 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:47.191698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f1ad6-6742-4b2b-b4f7-3a094696db3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:47.192344 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f1ad6-6742-4b2b-b4f7-3a094696db3e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:47.192630 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f1ad6-6742-4b2b-b4f7-3a094696db3e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:47.192987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f1ad6-6742-4b2b-b4f7-3a094696db3e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:47.206903 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f1ad6-6742-4b2b-b4f7-3a094696db3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:54:47.213009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-884f1ad6-6742-4b2b-b4f7-3a094696db3e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:54:47.213009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:54:47.213009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:54:47.213009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:54:47.213009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:54:47.213009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:54:47.213009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:54:47.213009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:54:47.213009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:54:47.213009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:54:47.213009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:54:47.213009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:54:47.213009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:54:47.213009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:54:47.213009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:54:47.213009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:54:47.214085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:54:47.214085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:54:47.214085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:54:47.214085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:54:47.214085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:54:47.214085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:54:47.214085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:54:47.214085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:54:47.214085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:54:47.214085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:54:47.214085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:54:47.214085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:54:47.214085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:54:47.214085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:54:47.214085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:54:47.214085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:54:47.215674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:54:47.215674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:54:47.215674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:54:47.215674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:54:47.215674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:54:47.215674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:54:47.215674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:54:47.215674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:54:47.215674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:54:47.215674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:54:47.215674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:54:47.215674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:54:47.215674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 456/913] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Oct 20 05:54:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:54:47.221183 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733a7d9-0ddd-4026-ad11-69a89ad67ad4 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:47.266153 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:47.342308 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733a7d9-0ddd-4026-ad11-69a89ad67ad4 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:47.383326 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6361114-b54d-4ea7-aec8-4aee6407bdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:47.384199 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6361114-b54d-4ea7-aec8-4aee6407bdca None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:47.384480 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6361114-b54d-4ea7-aec8-4aee6407bdca None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:47.384923 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6361114-b54d-4ea7-aec8-4aee6407bdca None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:47.385576 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6361114-b54d-4ea7-aec8-4aee6407bdca None admin] RBAC: Adding query filter params (name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:54:47.386010 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6361114-b54d-4ea7-aec8-4aee6407bdca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:47.388350 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6361114-b54d-4ea7-aec8-4aee6407bdca None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:47.397679 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 458/914] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Oct 20 05:54:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:54:47.403791 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f1ad6-6742-4b2b-b4f7-3a094696db3e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:47.461901 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:47.523134 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f1ad6-6742-4b2b-b4f7-3a094696db3e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:47.564835 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de095f8-4b70-4f24-8937-34e19a02fbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:47.565640 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de095f8-4b70-4f24-8937-34e19a02fbd1 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:47.566022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de095f8-4b70-4f24-8937-34e19a02fbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:47.566538 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de095f8-4b70-4f24-8937-34e19a02fbd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:47.578502 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de095f8-4b70-4f24-8937-34e19a02fbd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:47.580309 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de095f8-4b70-4f24-8937-34e19a02fbd1 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:47.596118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-8de095f8-4b70-4f24-8937-34e19a02fbd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:54:47.596118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:54:47.596118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:54:47.596118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:54:47.596118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:54:47.596118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:54:47.596118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:54:47.596118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:54:47.596118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:54:47.596118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:54:47.596118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:54:47.596118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:54:47.596118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:54:47.596118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:54:47.596118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:54:47.596118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:54:47.597347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:54:47.597347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:54:47.597347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:54:47.597347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:54:47.597347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:54:47.597347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:54:47.597347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:54:47.597347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:54:47.597347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:54:47.597347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:54:47.597347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:54:47.597347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:54:47.597347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:54:47.597347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:54:47.597347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:54:47.597347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:54:47.597347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:54:47.598272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:54:47.598272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:54:47.598272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:54:47.601009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 457/915] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:54:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:54:47.602991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6361114-b54d-4ea7-aec8-4aee6407bdca None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:47.710089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:47.712730 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6361114-b54d-4ea7-aec8-4aee6407bdca None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:47.750385 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890057f-3631-4ca7-a4a5-8b11082f28f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:47.751013 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890057f-3631-4ca7-a4a5-8b11082f28f3 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:47.751300 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890057f-3631-4ca7-a4a5-8b11082f28f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:47.751588 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890057f-3631-4ca7-a4a5-8b11082f28f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:47.752259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890057f-3631-4ca7-a4a5-8b11082f28f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:54:47.752567 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890057f-3631-4ca7-a4a5-8b11082f28f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:47.754111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890057f-3631-4ca7-a4a5-8b11082f28f3 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:47.767700 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 459/916] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:54:47.773630 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de095f8-4b70-4f24-8937-34e19a02fbd1 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:47.885423 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de095f8-4b70-4f24-8937-34e19a02fbd1 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:47.888464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:47.924278 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46934d16-33a1-45fc-90db-7837eeb4253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:47.924944 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46934d16-33a1-45fc-90db-7837eeb4253a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:47.925304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46934d16-33a1-45fc-90db-7837eeb4253a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:47.925610 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46934d16-33a1-45fc-90db-7837eeb4253a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:47.934202 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46934d16-33a1-45fc-90db-7837eeb4253a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:47.935775 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46934d16-33a1-45fc-90db-7837eeb4253a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:47.950228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-46934d16-33a1-45fc-90db-7837eeb4253a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 20 05:54:47.950228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:54:47.950228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:54:47.950228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:54:47.950228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:54:47.950228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:54:47.950228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:54:47.950228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:54:47.950228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:54:47.950228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:54:47.950228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:54:47.950228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:54:47.950228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:54:47.950228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:54:47.950228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:54:47.950228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:54:47.951513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:54:47.951513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:54:47.951513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:54:47.951513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:54:47.951513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:54:47.951513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:54:47.951513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:54:47.951513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:54:47.951513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:54:47.951513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:54:47.951513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:54:47.951513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:54:47.951513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:54:47.951513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:54:47.951513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:54:47.951513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:54:47.951513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:54:47.951513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:54:47.952511 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:54:47.952511 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:54:47.952511 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:54:47.952511 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:54:47.952511 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:54:47.952511 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:54:47.952511 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:54:47.952511 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:54:47.952511 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 20 05:54:47.952511 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:54:47.952511 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 458/917] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:54:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:54:47.957878 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890057f-3631-4ca7-a4a5-8b11082f28f3 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:48.071257 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890057f-3631-4ca7-a4a5-8b11082f28f3 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:48.110994 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839db4e0-a05d-4011-9ec6-e6983c392feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:48.111541 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839db4e0-a05d-4011-9ec6-e6983c392feb None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:48.111849 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839db4e0-a05d-4011-9ec6-e6983c392feb None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:48.112236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839db4e0-a05d-4011-9ec6-e6983c392feb None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:48.113217 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839db4e0-a05d-4011-9ec6-e6983c392feb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:54:48.113525 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839db4e0-a05d-4011-9ec6-e6983c392feb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:48.115281 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839db4e0-a05d-4011-9ec6-e6983c392feb None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:48.133387 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 460/918] 10.222.0.20 () {56 vars in 1143 bytes} [Thu Oct 20 05:54:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:54:48.140136 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46934d16-33a1-45fc-90db-7837eeb4253a None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:48.257672 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46934d16-33a1-45fc-90db-7837eeb4253a None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:48.303564 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b862ef9c-4593-4317-8b17-f83fb3168c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:48.304000 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862ef9c-4593-4317-8b17-f83fb3168c5c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:48.304334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862ef9c-4593-4317-8b17-f83fb3168c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:48.304608 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862ef9c-4593-4317-8b17-f83fb3168c5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:48.317950 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b862ef9c-4593-4317-8b17-f83fb3168c5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:48.319560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b862ef9c-4593-4317-8b17-f83fb3168c5c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:48.336002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-b862ef9c-4593-4317-8b17-f83fb3168c5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:54:48.336002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:54:48.336002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:54:48.336002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:54:48.336002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:54:48.336002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:54:48.336002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:54:48.336002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:54:48.336002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:54:48.336002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:54:48.336002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:54:48.336002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:54:48.336002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:54:48.336002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:54:48.336002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:54:48.336002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:54:48.337190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:54:48.337190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:54:48.337190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:54:48.337190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:54:48.337190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:54:48.337190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:54:48.337190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:54:48.337190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:54:48.337190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:54:48.337190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:54:48.337190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:54:48.337190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:54:48.337190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:54:48.337190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:54:48.337190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:54:48.337190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:54:48.337190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:54:48.338394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:54:48.338394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:54:48.338394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:54:48.341133 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 459/919] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:54:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:54:48.346822 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839db4e0-a05d-4011-9ec6-e6983c392feb None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:48.369771 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:48.472105 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839db4e0-a05d-4011-9ec6-e6983c392feb None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:48.514707 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338baeb8-2c6d-4047-bb8c-4009ba9b2957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:48.515391 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338baeb8-2c6d-4047-bb8c-4009ba9b2957 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:48.515711 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338baeb8-2c6d-4047-bb8c-4009ba9b2957 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:48.515997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338baeb8-2c6d-4047-bb8c-4009ba9b2957 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:48.516805 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338baeb8-2c6d-4047-bb8c-4009ba9b2957 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:54:48.517125 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338baeb8-2c6d-4047-bb8c-4009ba9b2957 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:48.519977 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338baeb8-2c6d-4047-bb8c-4009ba9b2957 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:48.532054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 461/920] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:54:48.537960 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b862ef9c-4593-4317-8b17-f83fb3168c5c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:48.555949 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:48.670493 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b862ef9c-4593-4317-8b17-f83fb3168c5c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:48.709973 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeac546-abad-4eac-bb37-ea90d3765f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:48.710605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeac546-abad-4eac-bb37-ea90d3765f44 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:48.710904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeac546-abad-4eac-bb37-ea90d3765f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:48.711146 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeac546-abad-4eac-bb37-ea90d3765f44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:48.718738 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeac546-abad-4eac-bb37-ea90d3765f44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:48.720210 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeac546-abad-4eac-bb37-ea90d3765f44 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:48.732080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-7aeac546-abad-4eac-bb37-ea90d3765f44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:54:48.732080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:54:48.732080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:54:48.732080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:54:48.732080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:54:48.732080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:54:48.732080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:54:48.732080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:54:48.732080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:54:48.732080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:54:48.732080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:54:48.732080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:54:48.732080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:54:48.732080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:54:48.732080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:54:48.733327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:54:48.733327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:54:48.733327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:54:48.733327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:54:48.733327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:54:48.733327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:54:48.733327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:54:48.733327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:54:48.733327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:54:48.733327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:54:48.733327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:54:48.733327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:54:48.733327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:54:48.733327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:54:48.733327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:54:48.733327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:54:48.733327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:54:48.734297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:54:48.734297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:54:48.734297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:54:48.734297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:54:48.734297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:54:48.734297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:54:48.734297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:54:48.734297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:54:48.738094 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 460/921] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:54:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:54:48.739116 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338baeb8-2c6d-4047-bb8c-4009ba9b2957 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:48.826118 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:48.850494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338baeb8-2c6d-4047-bb8c-4009ba9b2957 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:48.891234 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfc910-bb75-4656-9624-ee10b406ce4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:48.892053 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfc910-bb75-4656-9624-ee10b406ce4c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:48.892412 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfc910-bb75-4656-9624-ee10b406ce4c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:48.892800 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfc910-bb75-4656-9624-ee10b406ce4c None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:48.893823 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dfc910-bb75-4656-9624-ee10b406ce4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:54:48.894162 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dfc910-bb75-4656-9624-ee10b406ce4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:48.895839 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dfc910-bb75-4656-9624-ee10b406ce4c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:48.909796 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 462/922] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Oct 20 05:54:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:54:48.916437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeac546-abad-4eac-bb37-ea90d3765f44 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:49.007276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:49.036534 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeac546-abad-4eac-bb37-ea90d3765f44 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:49.073115 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03294b-e64a-49be-91e9-a633de530250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:49.074404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03294b-e64a-49be-91e9-a633de530250 None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:49.074866 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03294b-e64a-49be-91e9-a633de530250 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:49.075011 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03294b-e64a-49be-91e9-a633de530250 None admin] PATH_INFO: `/v3/projects/0ee65b9a8229495bbe97318f060d6342/users/cfbaaf32475a4c82a34948602d8f4e35/roles/afd859d2deed4d51aa3f6326920441c8` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:49.090927 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03294b-e64a-49be-91e9-a633de530250 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ee65b9a8229495bbe97318f060d6342, user_id=cfbaaf32475a4c82a34948602d8f4e35, role_id=afd859d2deed4d51aa3f6326920441c8)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:49.092396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03294b-e64a-49be-91e9-a633de530250 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:49.105508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 461/923] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:54:48 2022] PUT /identity/v3/projects/0ee65b9a8229495bbe97318f060d6342/users/cfbaaf32475a4c82a34948602d8f4e35/roles/afd859d2deed4d51aa3f6326920441c8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:54:51.267025 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdb17e-015c-41b2-8250-e16686418bcc None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:51.267628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdb17e-015c-41b2-8250-e16686418bcc None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:51.267628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdb17e-015c-41b2-8250-e16686418bcc None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:51.269065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 463/924] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:54:51.274051 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221e622-bcba-499e-875a-0ab2c4ad62cf None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:51.274470 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221e622-bcba-499e-875a-0ab2c4ad62cf None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:51.274603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221e622-bcba-499e-875a-0ab2c4ad62cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:51.294229 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-3221e622-bcba-499e-875a-0ab2c4ad62cf None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:54:51.295441 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3221e622-bcba-499e-875a-0ab2c4ad62cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:54:51.366930 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 462/925] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:54:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:54:51.644102 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdb17e-015c-41b2-8250-e16686418bcc None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:51.765312 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdb17e-015c-41b2-8250-e16686418bcc None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:51.805917 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160d315-275c-4c66-957a-6e1c4a8fa50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:51.806550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160d315-275c-4c66-957a-6e1c4a8fa50b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:51.806822 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160d315-275c-4c66-957a-6e1c4a8fa50b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:51.807091 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160d315-275c-4c66-957a-6e1c4a8fa50b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:51.814440 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:51.818323 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f160d315-275c-4c66-957a-6e1c4a8fa50b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:54:51.825538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-f160d315-275c-4c66-957a-6e1c4a8fa50b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:54:51.825538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:54:51.825538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:54:51.825538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:54:51.825538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:54:51.825538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:54:51.825538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:54:51.825538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:54:51.825538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:54:51.825538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:54:51.825538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:54:51.825538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:54:51.825538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:54:51.825538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:54:51.825538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:54:51.825538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:54:51.826672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:54:51.826672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:54:51.826672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:54:51.826672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:54:51.826672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:54:51.826672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:54:51.826672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:54:51.826672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:54:51.826672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:54:51.826672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:54:51.826672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:54:51.826672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:54:51.826672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:54:51.826672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:54:51.826672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:54:51.826672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:54:51.827627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:54:51.827627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:54:51.827627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:54:51.827627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:54:51.827627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:54:51.827627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:54:51.827627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:54:51.827627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:54:51.827627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:54:51.827627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:54:51.827627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 05:54:51.827627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:54:51.827627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 464/926] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Oct 20 05:54:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:54:51.833681 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221e622-bcba-499e-875a-0ab2c4ad62cf None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:51.892480 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:51.937806 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221e622-bcba-499e-875a-0ab2c4ad62cf None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:51.975388 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d227689-d9ac-46fa-b9b1-cb6d1457bb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:51.976100 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d227689-d9ac-46fa-b9b1-cb6d1457bb0e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:51.976396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d227689-d9ac-46fa-b9b1-cb6d1457bb0e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:51.976897 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d227689-d9ac-46fa-b9b1-cb6d1457bb0e None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:51.977485 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d227689-d9ac-46fa-b9b1-cb6d1457bb0e None admin] RBAC: Adding query filter params (name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:54:51.977816 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d227689-d9ac-46fa-b9b1-cb6d1457bb0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:51.979289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d227689-d9ac-46fa-b9b1-cb6d1457bb0e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:51.994274 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 463/927] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Oct 20 05:54:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:54:51.995796 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160d315-275c-4c66-957a-6e1c4a8fa50b None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:52.107037 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160d315-275c-4c66-957a-6e1c4a8fa50b None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:52.145439 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb07de-add6-4cb5-94f2-be3edf6740d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:52.146389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb07de-add6-4cb5-94f2-be3edf6740d0 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:52.146825 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb07de-add6-4cb5-94f2-be3edf6740d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:52.147090 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb07de-add6-4cb5-94f2-be3edf6740d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:52.158834 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb07de-add6-4cb5-94f2-be3edf6740d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:52.160304 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb07de-add6-4cb5-94f2-be3edf6740d0 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:52.177138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-03bb07de-add6-4cb5-94f2-be3edf6740d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:54:52.177138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:54:52.177138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:54:52.177138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:54:52.177138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:54:52.177138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:54:52.177138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:54:52.177138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:54:52.177138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:54:52.177138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:54:52.177138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:54:52.177138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:54:52.177138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:54:52.177138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:54:52.177138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:54:52.177138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:54:52.178578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:54:52.178578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:54:52.178578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:54:52.178578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:54:52.178578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:54:52.178578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:54:52.178578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:54:52.178578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:54:52.178578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:54:52.178578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:54:52.178578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:54:52.178578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:54:52.178578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:54:52.178578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:54:52.178578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:54:52.178578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:54:52.178578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:54:52.179636 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:54:52.179636 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:54:52.179636 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:54:52.184892 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 465/928] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:54:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:54:52.185901 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d227689-d9ac-46fa-b9b1-cb6d1457bb0e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:52.299741 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d227689-d9ac-46fa-b9b1-cb6d1457bb0e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:52.317823 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:52.336955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4fd83-153c-4b32-b966-bb20cbec0b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:52.337592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4fd83-153c-4b32-b966-bb20cbec0b19 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:52.337864 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4fd83-153c-4b32-b966-bb20cbec0b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:52.338130 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4fd83-153c-4b32-b966-bb20cbec0b19 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:52.338756 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c4fd83-153c-4b32-b966-bb20cbec0b19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:54:52.339057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c4fd83-153c-4b32-b966-bb20cbec0b19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:52.340399 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c4fd83-153c-4b32-b966-bb20cbec0b19 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:52.352034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 464/929] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:54:52.358483 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb07de-add6-4cb5-94f2-be3edf6740d0 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:52.409615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:52.461349 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb07de-add6-4cb5-94f2-be3edf6740d0 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:52.496034 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba23586-e79a-4848-8c95-9dbdf5a1c22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:52.496712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba23586-e79a-4848-8c95-9dbdf5a1c22a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:52.497001 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba23586-e79a-4848-8c95-9dbdf5a1c22a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:52.497268 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba23586-e79a-4848-8c95-9dbdf5a1c22a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:52.504851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba23586-e79a-4848-8c95-9dbdf5a1c22a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:52.506277 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba23586-e79a-4848-8c95-9dbdf5a1c22a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:52.518494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-1ba23586-e79a-4848-8c95-9dbdf5a1c22a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 20 05:54:52.518494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:54:52.518494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:54:52.518494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:54:52.518494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:54:52.518494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:54:52.518494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:54:52.518494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:54:52.518494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:54:52.518494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:54:52.518494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:54:52.518494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:54:52.518494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:54:52.518494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:54:52.518494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:54:52.518494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:54:52.519813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:54:52.519813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:54:52.519813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:54:52.519813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:54:52.519813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:54:52.519813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:54:52.519813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:54:52.519813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:54:52.519813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:54:52.519813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:54:52.519813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:54:52.519813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:54:52.519813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:54:52.519813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:54:52.519813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:54:52.519813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:54:52.519813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:54:52.519813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:54:52.521375 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:54:52.521375 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:54:52.521375 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:54:52.521375 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:54:52.521375 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:54:52.521375 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:54:52.521375 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:54:52.521375 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:54:52.521375 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 20 05:54:52.521375 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:54:52.521375 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 466/930] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:54:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:54:52.525394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4fd83-153c-4b32-b966-bb20cbec0b19 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:52.634745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4fd83-153c-4b32-b966-bb20cbec0b19 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:52.676827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f56137-faa2-4444-9087-031fab032f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:52.677576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f56137-faa2-4444-9087-031fab032f00 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:52.677955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f56137-faa2-4444-9087-031fab032f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:52.678229 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f56137-faa2-4444-9087-031fab032f00 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:52.679131 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f56137-faa2-4444-9087-031fab032f00 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:54:52.679308 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f56137-faa2-4444-9087-031fab032f00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:52.680862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f56137-faa2-4444-9087-031fab032f00 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:52.700641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 465/931] 10.222.0.20 () {56 vars in 1143 bytes} [Thu Oct 20 05:54:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:54:52.706337 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba23586-e79a-4848-8c95-9dbdf5a1c22a None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:52.813844 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba23586-e79a-4848-8c95-9dbdf5a1c22a None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:52.825314 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:52.853675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5064e0-49b5-453d-ab0e-33187abf836e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:52.854296 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5064e0-49b5-453d-ab0e-33187abf836e None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:52.854604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5064e0-49b5-453d-ab0e-33187abf836e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:52.855483 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5064e0-49b5-453d-ab0e-33187abf836e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:52.929908 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5064e0-49b5-453d-ab0e-33187abf836e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:52.931260 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5064e0-49b5-453d-ab0e-33187abf836e None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:53.041304 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 467/932] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:54:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 05:54:53.048137 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f56137-faa2-4444-9087-031fab032f00 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:53.094441 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:53.165881 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f56137-faa2-4444-9087-031fab032f00 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:53.204620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb70f87-f007-44ef-9a8c-5b3a2b275176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:53.205194 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb70f87-f007-44ef-9a8c-5b3a2b275176 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:53.205529 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb70f87-f007-44ef-9a8c-5b3a2b275176 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:53.205767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb70f87-f007-44ef-9a8c-5b3a2b275176 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:53.216752 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb70f87-f007-44ef-9a8c-5b3a2b275176 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:53.218328 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb70f87-f007-44ef-9a8c-5b3a2b275176 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:53.236411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-fbb70f87-f007-44ef-9a8c-5b3a2b275176 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:54:53.236411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:54:53.236411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:54:53.236411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:54:53.236411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:54:53.236411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:54:53.236411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:54:53.236411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:54:53.236411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:54:53.236411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:54:53.236411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:54:53.236411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:54:53.236411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:54:53.236411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:54:53.236411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:54:53.236411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:54:53.237666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:54:53.237666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:54:53.237666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:54:53.237666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:54:53.237666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:54:53.237666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:54:53.237666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:54:53.237666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:54:53.237666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:54:53.237666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:54:53.237666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:54:53.237666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:54:53.237666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:54:53.237666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:54:53.237666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:54:53.237666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:54:53.237666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:54:53.238625 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:54:53.238625 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:54:53.238625 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:54:53.238625 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 466/933] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:54:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:54:53.243653 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5064e0-49b5-453d-ab0e-33187abf836e None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:53.257262 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:53.361685 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5064e0-49b5-453d-ab0e-33187abf836e None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:53.401041 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b0877-a886-4c2e-9d15-00ec145d2db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:53.401799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b0877-a886-4c2e-9d15-00ec145d2db1 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:53.402076 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b0877-a886-4c2e-9d15-00ec145d2db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:53.402343 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b0877-a886-4c2e-9d15-00ec145d2db1 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:53.402970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b0877-a886-4c2e-9d15-00ec145d2db1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:54:53.403266 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b0877-a886-4c2e-9d15-00ec145d2db1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:53.404647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b0877-a886-4c2e-9d15-00ec145d2db1 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:53.417224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 468/934] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:54:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:54:53.422768 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb70f87-f007-44ef-9a8c-5b3a2b275176 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:53.518035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:53.536290 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb70f87-f007-44ef-9a8c-5b3a2b275176 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:53.576820 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3fd5a-7844-427b-8fc1-cab834208da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:53.577478 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3fd5a-7844-427b-8fc1-cab834208da8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:53.577822 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3fd5a-7844-427b-8fc1-cab834208da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:53.578223 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3fd5a-7844-427b-8fc1-cab834208da8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:53.586866 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe3fd5a-7844-427b-8fc1-cab834208da8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:53.588522 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe3fd5a-7844-427b-8fc1-cab834208da8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:53.603636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-3fe3fd5a-7844-427b-8fc1-cab834208da8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:54:53.603636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:54:53.603636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:54:53.603636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:54:53.603636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:54:53.603636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:54:53.603636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:54:53.603636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:54:53.603636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:54:53.603636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:54:53.603636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:54:53.603636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:54:53.603636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:54:53.603636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:54:53.603636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:54:53.605302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:54:53.605302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:54:53.605302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:54:53.605302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:54:53.605302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:54:53.605302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:54:53.605302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:54:53.605302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:54:53.605302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:54:53.605302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:54:53.605302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:54:53.605302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:54:53.605302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:54:53.605302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:54:53.605302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:54:53.605302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:54:53.605302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:54:53.606382 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:54:53.606382 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:54:53.606382 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:54:53.606382 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:54:53.606382 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:54:53.606382 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:54:53.606382 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:54:53.606382 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:54:53.606382 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 467/935] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:54:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:54:53.611843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b0877-a886-4c2e-9d15-00ec145d2db1 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:53.692531 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:53.730975 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b0877-a886-4c2e-9d15-00ec145d2db1 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:53.771633 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa65e7-4aa4-4ca9-a6dc-c08247066b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:53.772364 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa65e7-4aa4-4ca9-a6dc-c08247066b05 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:53.772887 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa65e7-4aa4-4ca9-a6dc-c08247066b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:53.773235 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa65e7-4aa4-4ca9-a6dc-c08247066b05 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:53.773991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa65e7-4aa4-4ca9-a6dc-c08247066b05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:54:53.774368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa65e7-4aa4-4ca9-a6dc-c08247066b05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:53.776199 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa65e7-4aa4-4ca9-a6dc-c08247066b05 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:53.790400 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 469/936] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Oct 20 05:54:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:54:53.796971 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3fd5a-7844-427b-8fc1-cab834208da8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:53.908777 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3fd5a-7844-427b-8fc1-cab834208da8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:53.944385 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfec7f-45c9-446e-8c12-89d869ba7e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:53.945028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfec7f-45c9-446e-8c12-89d869ba7e63 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:53.945303 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfec7f-45c9-446e-8c12-89d869ba7e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:53.945569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfec7f-45c9-446e-8c12-89d869ba7e63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:53.946330 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfec7f-45c9-446e-8c12-89d869ba7e63 None admin] RBAC: Adding query filter params (role.id=afd859d2deed4d51aa3f6326920441c8, scope.project.id=0ee65b9a8229495bbe97318f060d6342, user.id=cfbaaf32475a4c82a34948602d8f4e35) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:54:53.946759 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfec7f-45c9-446e-8c12-89d869ba7e63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:53.948911 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfec7f-45c9-446e-8c12-89d869ba7e63 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:53.964726 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 468/937] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:54:53 2022] GET /identity/v3/role_assignments?user.id=cfbaaf32475a4c82a34948602d8f4e35&scope.project.id=0ee65b9a8229495bbe97318f060d6342&role.id=afd859d2deed4d51aa3f6326920441c8 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:54:56.171880 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b906b0-3036-4e27-86d2-f0ad91f4ef01 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:56.172432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b906b0-3036-4e27-86d2-f0ad91f4ef01 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:56.172432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b906b0-3036-4e27-86d2-f0ad91f4ef01 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:56.173777 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 470/938] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:54:56.178916 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf562a8b-ce84-4935-99dd-036c4ff97a4a None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:56.179529 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf562a8b-ce84-4935-99dd-036c4ff97a4a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:56.179529 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf562a8b-ce84-4935-99dd-036c4ff97a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:56.198085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-bf562a8b-ce84-4935-99dd-036c4ff97a4a None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:54:56.199452 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-bf562a8b-ce84-4935-99dd-036c4ff97a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:54:56.202184 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:56.267031 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 469/939] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:54:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:54:56.555418 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b906b0-3036-4e27-86d2-f0ad91f4ef01 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:56.667818 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b906b0-3036-4e27-86d2-f0ad91f4ef01 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:56.698887 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:56.705401 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb943f13-681f-43f3-b927-5cef4c6d5af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:56.706029 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb943f13-681f-43f3-b927-5cef4c6d5af0 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:56.706302 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb943f13-681f-43f3-b927-5cef4c6d5af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:56.706570 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb943f13-681f-43f3-b927-5cef4c6d5af0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:56.716546 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb943f13-681f-43f3-b927-5cef4c6d5af0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:54:56.721705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-cb943f13-681f-43f3-b927-5cef4c6d5af0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:54:56.721705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:54:56.721705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:54:56.721705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:54:56.721705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:54:56.721705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:54:56.721705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:54:56.721705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:54:56.721705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:54:56.721705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:54:56.721705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:54:56.721705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:54:56.721705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:54:56.721705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:54:56.721705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:54:56.721705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:54:56.722843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:54:56.722843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:54:56.722843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:54:56.722843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:54:56.722843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:54:56.722843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:54:56.722843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:54:56.722843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:54:56.722843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:54:56.722843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:54:56.722843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:54:56.722843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:54:56.722843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:54:56.722843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:54:56.722843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:54:56.722843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:54:56.723695 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:54:56.723695 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:54:56.723695 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:54:56.723695 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:54:56.723695 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:54:56.723695 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:54:56.723695 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:54:56.723695 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:54:56.723695 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:54:56.723695 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:54:56.723695 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:54:56.723695 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:54:56.723695 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 471/940] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Oct 20 05:54:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:54:56.730050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf562a8b-ce84-4935-99dd-036c4ff97a4a None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:56.838352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf562a8b-ce84-4935-99dd-036c4ff97a4a None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:56.880065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1401a-cfca-4b56-b606-c22522b8d7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:56.880816 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1401a-cfca-4b56-b606-c22522b8d7d8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:56.881146 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1401a-cfca-4b56-b606-c22522b8d7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:56.881458 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1401a-cfca-4b56-b606-c22522b8d7d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:56.882328 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b1401a-cfca-4b56-b606-c22522b8d7d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:54:56.882694 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b1401a-cfca-4b56-b606-c22522b8d7d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:56.884408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b1401a-cfca-4b56-b606-c22522b8d7d8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:56.893895 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 470/941] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:54:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:54:56.899856 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb943f13-681f-43f3-b927-5cef4c6d5af0 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:57.010635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb943f13-681f-43f3-b927-5cef4c6d5af0 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:57.051872 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ba170-b934-40d5-a280-e7d17bf57d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:57.052505 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ba170-b934-40d5-a280-e7d17bf57d45 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:57.052886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ba170-b934-40d5-a280-e7d17bf57d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:57.053146 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ba170-b934-40d5-a280-e7d17bf57d45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:57.065917 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ba170-b934-40d5-a280-e7d17bf57d45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:57.067532 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ba170-b934-40d5-a280-e7d17bf57d45 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:57.083859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-0c9ba170-b934-40d5-a280-e7d17bf57d45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:54:57.083859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:54:57.083859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:54:57.083859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:54:57.083859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:54:57.083859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:54:57.083859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:54:57.083859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:54:57.083859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:54:57.083859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:54:57.083859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:54:57.083859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:54:57.083859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:54:57.083859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:54:57.083859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:54:57.083859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:54:57.085111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:54:57.085111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:54:57.085111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:54:57.085111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:54:57.085111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:54:57.085111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:54:57.085111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:54:57.085111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:54:57.085111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:54:57.085111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:54:57.085111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:54:57.085111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:54:57.085111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:54:57.085111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:54:57.085111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:54:57.085111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:54:57.085111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:54:57.086368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:54:57.086368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:54:57.086368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:54:57.086368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 472/942] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:54:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:54:57.091818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1401a-cfca-4b56-b606-c22522b8d7d8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:57.094982 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:57.212141 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1401a-cfca-4b56-b606-c22522b8d7d8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:57.252043 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9d5d2-4ef8-424f-b766-ac617fc6644c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:57.252672 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9d5d2-4ef8-424f-b766-ac617fc6644c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:57.253015 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9d5d2-4ef8-424f-b766-ac617fc6644c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:57.253345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9d5d2-4ef8-424f-b766-ac617fc6644c None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:57.254014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a9d5d2-4ef8-424f-b766-ac617fc6644c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:54:57.254336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a9d5d2-4ef8-424f-b766-ac617fc6644c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:57.255906 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a9d5d2-4ef8-424f-b766-ac617fc6644c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:57.270864 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 471/943] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:54:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:54:57.276643 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ba170-b934-40d5-a280-e7d17bf57d45 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:57.326508 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:57.387361 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ba170-b934-40d5-a280-e7d17bf57d45 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:57.423775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8105e4-87b3-43fa-942a-3f7191a1178b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:57.424434 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8105e4-87b3-43fa-942a-3f7191a1178b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:57.424815 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8105e4-87b3-43fa-942a-3f7191a1178b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:57.425099 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8105e4-87b3-43fa-942a-3f7191a1178b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:57.432997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8105e4-87b3-43fa-942a-3f7191a1178b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:57.434867 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8105e4-87b3-43fa-942a-3f7191a1178b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:57.447431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-2d8105e4-87b3-43fa-942a-3f7191a1178b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 20 05:54:57.447431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:54:57.447431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:54:57.447431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:54:57.447431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:54:57.447431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:54:57.447431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:54:57.447431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:54:57.447431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:54:57.447431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:54:57.447431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:54:57.447431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:54:57.447431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:54:57.447431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:54:57.447431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:54:57.447431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:54:57.449340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:54:57.449340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:54:57.449340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:54:57.449340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:54:57.449340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:54:57.449340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:54:57.449340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:54:57.449340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:54:57.449340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:54:57.449340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:54:57.449340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:54:57.449340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:54:57.449340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:54:57.449340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:54:57.449340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:54:57.449340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:54:57.449340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:54:57.449340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:54:57.450456 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:54:57.450456 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:54:57.450456 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:54:57.450456 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:54:57.450456 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:54:57.450456 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:54:57.450456 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:54:57.450456 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:54:57.450456 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 20 05:54:57.450456 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:54:57.450456 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 473/944] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:54:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:54:57.455447 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9d5d2-4ef8-424f-b766-ac617fc6644c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:57.524755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:57.566127 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9d5d2-4ef8-424f-b766-ac617fc6644c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:57.606208 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601bb203-63a8-40da-a7f5-fb2db3317197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:57.606827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bb203-63a8-40da-a7f5-fb2db3317197 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:57.607105 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bb203-63a8-40da-a7f5-fb2db3317197 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:57.607375 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bb203-63a8-40da-a7f5-fb2db3317197 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:57.608090 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601bb203-63a8-40da-a7f5-fb2db3317197 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:54:57.608393 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601bb203-63a8-40da-a7f5-fb2db3317197 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:57.609852 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601bb203-63a8-40da-a7f5-fb2db3317197 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:57.628342 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 472/945] 10.222.0.20 () {56 vars in 1143 bytes} [Thu Oct 20 05:54:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:54:57.634460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8105e4-87b3-43fa-942a-3f7191a1178b None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:57.741724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8105e4-87b3-43fa-942a-3f7191a1178b None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:57.743178 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:57.781559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32114f5c-9939-4a19-a176-9ec18e57d01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:57.782196 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32114f5c-9939-4a19-a176-9ec18e57d01a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:57.782475 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32114f5c-9939-4a19-a176-9ec18e57d01a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:57.782746 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32114f5c-9939-4a19-a176-9ec18e57d01a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:57.797653 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32114f5c-9939-4a19-a176-9ec18e57d01a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:57.799182 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32114f5c-9939-4a19-a176-9ec18e57d01a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:57.817014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-32114f5c-9939-4a19-a176-9ec18e57d01a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:54:57.817014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:54:57.817014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:54:57.817014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:54:57.817014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:54:57.817014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:54:57.817014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:54:57.817014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:54:57.817014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:54:57.817014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:54:57.817014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:54:57.817014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:54:57.817014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:54:57.817014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:54:57.817014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:54:57.817014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:54:57.818250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:54:57.818250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:54:57.818250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:54:57.818250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:54:57.818250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:54:57.818250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:54:57.818250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:54:57.818250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:54:57.818250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:54:57.818250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:54:57.818250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:54:57.818250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:54:57.818250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:54:57.818250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:54:57.818250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:54:57.818250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:54:57.818250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:54:57.819169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:54:57.819169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:54:57.819169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:54:57.819169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 474/946] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:54:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:54:57.824782 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601bb203-63a8-40da-a7f5-fb2db3317197 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:57.934471 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601bb203-63a8-40da-a7f5-fb2db3317197 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:57.957236 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:57.973385 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902f114-4f9d-43c1-ad09-9fbc7d4867aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:57.974095 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902f114-4f9d-43c1-ad09-9fbc7d4867aa None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:57.974416 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902f114-4f9d-43c1-ad09-9fbc7d4867aa None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:57.974722 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902f114-4f9d-43c1-ad09-9fbc7d4867aa None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:57.975420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902f114-4f9d-43c1-ad09-9fbc7d4867aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:54:57.975752 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902f114-4f9d-43c1-ad09-9fbc7d4867aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:57.977337 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902f114-4f9d-43c1-ad09-9fbc7d4867aa None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:57.996226 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 473/947] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:54:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:54:57.997561 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32114f5c-9939-4a19-a176-9ec18e57d01a None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:58.123668 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32114f5c-9939-4a19-a176-9ec18e57d01a None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:58.165202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80439a9d-eba9-4075-8171-1f6fa31ae5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:58.165833 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80439a9d-eba9-4075-8171-1f6fa31ae5ce None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:58.166106 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80439a9d-eba9-4075-8171-1f6fa31ae5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:58.166371 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80439a9d-eba9-4075-8171-1f6fa31ae5ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:58.173959 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80439a9d-eba9-4075-8171-1f6fa31ae5ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:58.175581 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80439a9d-eba9-4075-8171-1f6fa31ae5ce None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:58.188657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-80439a9d-eba9-4075-8171-1f6fa31ae5ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:54:58.188657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:54:58.188657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:54:58.188657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:54:58.188657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:54:58.188657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:54:58.188657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:54:58.188657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:54:58.188657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:54:58.188657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:54:58.188657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:54:58.188657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:54:58.188657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:54:58.188657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:54:58.188657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:54:58.189851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:54:58.189851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:54:58.189851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:54:58.189851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:54:58.189851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:54:58.189851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:54:58.189851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:54:58.189851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:54:58.189851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:54:58.189851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:54:58.189851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:54:58.189851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:54:58.189851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:54:58.189851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:54:58.189851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:54:58.189851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:54:58.189851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:54:58.190767 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:54:58.190767 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:54:58.190767 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:54:58.190767 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:54:58.190767 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:54:58.190767 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:54:58.190767 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:54:58.190767 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:54:58.190767 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 475/948] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:54:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:54:58.195801 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902f114-4f9d-43c1-ad09-9fbc7d4867aa None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:58.307133 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902f114-4f9d-43c1-ad09-9fbc7d4867aa None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:58.343182 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95241d13-e25a-4a6a-8f41-1bbfbc860447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:58.343802 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95241d13-e25a-4a6a-8f41-1bbfbc860447 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:58.344069 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95241d13-e25a-4a6a-8f41-1bbfbc860447 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:58.344339 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95241d13-e25a-4a6a-8f41-1bbfbc860447 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:58.345128 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95241d13-e25a-4a6a-8f41-1bbfbc860447 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:54:58.345390 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95241d13-e25a-4a6a-8f41-1bbfbc860447 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:58.346998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95241d13-e25a-4a6a-8f41-1bbfbc860447 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:58.360461 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 474/949] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Oct 20 05:54:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:54:58.366497 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80439a9d-eba9-4075-8171-1f6fa31ae5ce None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:54:58.385009 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:54:58.471451 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80439a9d-eba9-4075-8171-1f6fa31ae5ce None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:54:58.506777 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146cd49e-a0e0-4f38-b92e-1dea353bb580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:54:58.507438 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146cd49e-a0e0-4f38-b92e-1dea353bb580 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:54:58.507714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146cd49e-a0e0-4f38-b92e-1dea353bb580 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:54:58.507987 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146cd49e-a0e0-4f38-b92e-1dea353bb580 None admin] PATH_INFO: `/v3/projects/0ee65b9a8229495bbe97318f060d6342/users/cfbaaf32475a4c82a34948602d8f4e35/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:54:58.510189 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146cd49e-a0e0-4f38-b92e-1dea353bb580 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ee65b9a8229495bbe97318f060d6342, user_id=cfbaaf32475a4c82a34948602d8f4e35, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:54:58.512272 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146cd49e-a0e0-4f38-b92e-1dea353bb580 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:54:58.524110 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 476/950] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Oct 20 05:54:58 2022] PUT /identity/v3/projects/0ee65b9a8229495bbe97318f060d6342/users/cfbaaf32475a4c82a34948602d8f4e35/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 05:55:00.930119 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70769c9-082f-4fc1-a4dc-8abd00db1678 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:00.930752 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70769c9-082f-4fc1-a4dc-8abd00db1678 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:00.930752 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70769c9-082f-4fc1-a4dc-8abd00db1678 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:00.932303 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 475/951] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:55:00.938301 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69eee2-c34a-4f30-be44-2cb7d90c9230 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:00.938739 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69eee2-c34a-4f30-be44-2cb7d90c9230 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:00.938899 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69eee2-c34a-4f30-be44-2cb7d90c9230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:00.958286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-bc69eee2-c34a-4f30-be44-2cb7d90c9230 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:55:00.959551 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-bc69eee2-c34a-4f30-be44-2cb7d90c9230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:55:01.044200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 477/952] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:55:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 05:55:01.335796 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70769c9-082f-4fc1-a4dc-8abd00db1678 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:55:01.384446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:55:01.450298 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70769c9-082f-4fc1-a4dc-8abd00db1678 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:55:01.485317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9b2b6-97b7-4c21-aff8-9d10204fbe34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:55:01.485944 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9b2b6-97b7-4c21-aff8-9d10204fbe34 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:01.486220 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9b2b6-97b7-4c21-aff8-9d10204fbe34 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:01.486489 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9b2b6-97b7-4c21-aff8-9d10204fbe34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:01.497708 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad9b2b6-97b7-4c21-aff8-9d10204fbe34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 05:55:01.503071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-fad9b2b6-97b7-4c21-aff8-9d10204fbe34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:55:01.503071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:55:01.503071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:55:01.503071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:55:01.503071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:55:01.503071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:55:01.503071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:55:01.503071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:55:01.503071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:55:01.503071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:55:01.503071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:55:01.503071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:55:01.503071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 05:55:01.503071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 05:55:01.503071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 05:55:01.503071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 05:55:01.504195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 05:55:01.504195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 05:55:01.504195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 05:55:01.504195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 05:55:01.504195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:55:01.504195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:55:01.504195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:55:01.504195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:55:01.504195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:55:01.504195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:55:01.504195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:55:01.504195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:55:01.504195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:55:01.504195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:55:01.504195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:55:01.504195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:55:01.505426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:55:01.505426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:55:01.505426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:55:01.505426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:55:01.505426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 05:55:01.505426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 05:55:01.505426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 05:55:01.505426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 05:55:01.505426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 05:55:01.505426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 05:55:01.505426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 05:55:01.505426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:55:01.509595 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 476/953] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Oct 20 05:55:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 05:55:01.511476 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69eee2-c34a-4f30-be44-2cb7d90c9230 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:55:01.515198 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:55:01.622207 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69eee2-c34a-4f30-be44-2cb7d90c9230 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:55:01.663091 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea391e2-ff7b-4d72-9e2c-af153259035f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:55:01.663703 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea391e2-ff7b-4d72-9e2c-af153259035f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:01.664119 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea391e2-ff7b-4d72-9e2c-af153259035f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:01.664502 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea391e2-ff7b-4d72-9e2c-af153259035f None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:01.665316 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea391e2-ff7b-4d72-9e2c-af153259035f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:55:01.665617 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea391e2-ff7b-4d72-9e2c-af153259035f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:55:01.667432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea391e2-ff7b-4d72-9e2c-af153259035f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:55:01.677467 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 478/954] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Oct 20 05:55:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:55:01.685340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9b2b6-97b7-4c21-aff8-9d10204fbe34 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:55:01.794713 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:55:01.798374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9b2b6-97b7-4c21-aff8-9d10204fbe34 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:55:01.836564 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d565f-eeb1-4202-8585-66ef2f0f47b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:55:01.837226 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d565f-eeb1-4202-8585-66ef2f0f47b0 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:01.837502 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d565f-eeb1-4202-8585-66ef2f0f47b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:01.837770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d565f-eeb1-4202-8585-66ef2f0f47b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:01.853663 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d565f-eeb1-4202-8585-66ef2f0f47b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:55:01.855397 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d565f-eeb1-4202-8585-66ef2f0f47b0 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:55:01.872857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-693d565f-eeb1-4202-8585-66ef2f0f47b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:55:01.872857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:55:01.872857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:55:01.872857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:55:01.872857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:55:01.872857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:55:01.872857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:55:01.872857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:55:01.872857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:55:01.872857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:55:01.872857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:55:01.872857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:55:01.872857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:55:01.872857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:55:01.872857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:55:01.872857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:55:01.874238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:55:01.874238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:55:01.874238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:55:01.874238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:55:01.874238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:55:01.874238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:55:01.874238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:55:01.874238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:55:01.874238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:55:01.874238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:55:01.874238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:55:01.874238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:55:01.874238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:55:01.874238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:55:01.874238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:55:01.874238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:55:01.874238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:55:01.875327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:55:01.875327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:55:01.875327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:55:01.875327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 477/955] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:55:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:55:01.880148 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea391e2-ff7b-4d72-9e2c-af153259035f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:55:01.952846 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:55:01.994750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea391e2-ff7b-4d72-9e2c-af153259035f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:55:02.034634 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2a7e3-b5bd-4b39-8e2a-0a0fa5a2e3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:55:02.035336 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2a7e3-b5bd-4b39-8e2a-0a0fa5a2e3d8 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:02.035609 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2a7e3-b5bd-4b39-8e2a-0a0fa5a2e3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:02.035877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2a7e3-b5bd-4b39-8e2a-0a0fa5a2e3d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:02.036616 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2a7e3-b5bd-4b39-8e2a-0a0fa5a2e3d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:55:02.037058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2a7e3-b5bd-4b39-8e2a-0a0fa5a2e3d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:55:02.038580 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2a7e3-b5bd-4b39-8e2a-0a0fa5a2e3d8 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:55:02.053110 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 479/956] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 05:55:02.059783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d565f-eeb1-4202-8585-66ef2f0f47b0 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:55:02.169349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d565f-eeb1-4202-8585-66ef2f0f47b0 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:55:02.206035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ca123-9e4d-4017-8af8-cc467a00164b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:55:02.206678 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ca123-9e4d-4017-8af8-cc467a00164b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:02.206955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ca123-9e4d-4017-8af8-cc467a00164b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:02.207223 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ca123-9e4d-4017-8af8-cc467a00164b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:02.215413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207ca123-9e4d-4017-8af8-cc467a00164b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:55:02.216910 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207ca123-9e4d-4017-8af8-cc467a00164b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:55:02.229907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-207ca123-9e4d-4017-8af8-cc467a00164b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 20 05:55:02.229907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:55:02.229907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:55:02.229907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:55:02.229907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:55:02.229907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:55:02.229907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:55:02.229907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:55:02.229907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:55:02.229907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:55:02.229907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:55:02.229907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:55:02.229907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 05:55:02.229907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 05:55:02.229907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 05:55:02.229907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 05:55:02.231224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:55:02.231224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:55:02.231224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 05:55:02.231224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:55:02.231224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 05:55:02.231224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 05:55:02.231224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:55:02.231224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:55:02.231224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:55:02.231224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:55:02.231224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:55:02.231224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:55:02.231224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:55:02.231224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:55:02.231224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:55:02.231224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:55:02.231224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:55:02.231224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:55:02.232358 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:55:02.232358 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:55:02.232358 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 05:55:02.232358 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 05:55:02.232358 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 05:55:02.232358 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 05:55:02.232358 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 05:55:02.232358 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 05:55:02.232358 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 20 05:55:02.232358 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:55:02.232358 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 478/957] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:55:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:55:02.237038 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2a7e3-b5bd-4b39-8e2a-0a0fa5a2e3d8 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:55:02.345736 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2a7e3-b5bd-4b39-8e2a-0a0fa5a2e3d8 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:55:02.372214 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:55:02.381447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c3f42-b673-4ff5-8cbd-02110eed1f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:55:02.382063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c3f42-b673-4ff5-8cbd-02110eed1f60 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:02.382335 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c3f42-b673-4ff5-8cbd-02110eed1f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:02.382602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c3f42-b673-4ff5-8cbd-02110eed1f60 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:02.383325 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4c3f42-b673-4ff5-8cbd-02110eed1f60 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:55:02.383877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4c3f42-b673-4ff5-8cbd-02110eed1f60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:55:02.385397 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4c3f42-b673-4ff5-8cbd-02110eed1f60 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:55:02.405227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 480/958] 10.222.0.20 () {56 vars in 1143 bytes} [Thu Oct 20 05:55:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:55:02.410986 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ca123-9e4d-4017-8af8-cc467a00164b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:55:02.427494 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:55:02.516235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ca123-9e4d-4017-8af8-cc467a00164b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:55:02.550325 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2add3-558f-46a4-bd91-da0f48d28fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:55:02.550938 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2add3-558f-46a4-bd91-da0f48d28fbb None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:02.551211 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2add3-558f-46a4-bd91-da0f48d28fbb None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:02.551478 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2add3-558f-46a4-bd91-da0f48d28fbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:02.618964 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f2add3-558f-46a4-bd91-da0f48d28fbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:55:02.620960 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f2add3-558f-46a4-bd91-da0f48d28fbb None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:55:02.634644 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:55:02.719954 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 479/959] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Oct 20 05:55:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 05:55:02.725931 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c3f42-b673-4ff5-8cbd-02110eed1f60 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:55:02.821028 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c3f42-b673-4ff5-8cbd-02110eed1f60 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:55:02.852805 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d459d55-dbc5-4eb6-a111-92c7c00d7979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:55:02.853417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d459d55-dbc5-4eb6-a111-92c7c00d7979 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:02.853690 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d459d55-dbc5-4eb6-a111-92c7c00d7979 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:02.853953 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d459d55-dbc5-4eb6-a111-92c7c00d7979 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:02.864894 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d459d55-dbc5-4eb6-a111-92c7c00d7979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:55:02.866325 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d459d55-dbc5-4eb6-a111-92c7c00d7979 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:55:02.884344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-0d459d55-dbc5-4eb6-a111-92c7c00d7979 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:55:02.884344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:55:02.884344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:55:02.884344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:55:02.884344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:55:02.884344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:55:02.884344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:55:02.884344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:55:02.884344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:55:02.884344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:55:02.884344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:55:02.884344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:55:02.884344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 05:55:02.884344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 05:55:02.884344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 05:55:02.884344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 05:55:02.885553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:55:02.885553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:55:02.885553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:55:02.885553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:55:02.885553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:55:02.885553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:55:02.885553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:55:02.885553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:55:02.885553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:55:02.885553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:55:02.885553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:55:02.885553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:55:02.885553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:55:02.885553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:55:02.885553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:55:02.885553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:55:02.885553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 05:55:02.886734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 05:55:02.886734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 05:55:02.886734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:55:02.886734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 481/960] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:55:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:55:02.892260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2add3-558f-46a4-bd91-da0f48d28fbb None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:55:03.019565 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2add3-558f-46a4-bd91-da0f48d28fbb None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:55:03.032678 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:55:03.054395 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066bb06-59fa-47eb-889c-7ecfce79d0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:55:03.054990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066bb06-59fa-47eb-889c-7ecfce79d0ab None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:03.055265 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066bb06-59fa-47eb-889c-7ecfce79d0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:03.055532 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066bb06-59fa-47eb-889c-7ecfce79d0ab None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:03.056157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066bb06-59fa-47eb-889c-7ecfce79d0ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:55:03.056453 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066bb06-59fa-47eb-889c-7ecfce79d0ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:55:03.058054 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066bb06-59fa-47eb-889c-7ecfce79d0ab None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:55:03.069181 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 480/961] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Oct 20 05:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:55:03.074878 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d459d55-dbc5-4eb6-a111-92c7c00d7979 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:55:03.109277 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:55:03.173654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d459d55-dbc5-4eb6-a111-92c7c00d7979 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:55:03.206396 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556ffc9-aeb6-411b-832a-f327a3e9c546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:55:03.207031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556ffc9-aeb6-411b-832a-f327a3e9c546 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:03.207306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556ffc9-aeb6-411b-832a-f327a3e9c546 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:03.207572 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556ffc9-aeb6-411b-832a-f327a3e9c546 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:03.213912 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4556ffc9-aeb6-411b-832a-f327a3e9c546 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:55:03.215317 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4556ffc9-aeb6-411b-832a-f327a3e9c546 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:55:03.225944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-4556ffc9-aeb6-411b-832a-f327a3e9c546 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:55:03.225944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:55:03.225944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:55:03.225944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:55:03.225944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:55:03.225944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:55:03.225944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:55:03.225944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:55:03.225944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:55:03.225944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:55:03.225944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:55:03.225944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:55:03.225944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 05:55:03.225944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 05:55:03.225944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 05:55:03.226957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 05:55:03.226957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:55:03.226957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:55:03.226957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:55:03.226957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:55:03.226957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:55:03.226957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:55:03.226957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:55:03.226957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 05:55:03.226957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:55:03.226957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:55:03.226957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:55:03.226957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:55:03.226957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:55:03.226957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:55:03.226957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:55:03.226957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:55:03.228073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 05:55:03.228073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 05:55:03.228073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 05:55:03.228073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 05:55:03.228073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 05:55:03.228073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 05:55:03.228073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 05:55:03.228073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 05:55:03.228073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 482/962] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Oct 20 05:55:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:55:03.232795 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066bb06-59fa-47eb-889c-7ecfce79d0ab None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:55:03.334324 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066bb06-59fa-47eb-889c-7ecfce79d0ab None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:55:03.366102 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cdfb80-75e7-4efa-ab4a-5f1247c85df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:55:03.366698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cdfb80-75e7-4efa-ab4a-5f1247c85df6 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:03.366979 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cdfb80-75e7-4efa-ab4a-5f1247c85df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:03.367250 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cdfb80-75e7-4efa-ab4a-5f1247c85df6 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:03.367925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cdfb80-75e7-4efa-ab4a-5f1247c85df6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:55:03.368229 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cdfb80-75e7-4efa-ab4a-5f1247c85df6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:55:03.370149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cdfb80-75e7-4efa-ab4a-5f1247c85df6 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:55:03.382312 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 481/963] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Oct 20 05:55:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:55:03.389557 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556ffc9-aeb6-411b-832a-f327a3e9c546 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:55:03.480603 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:55:03.486771 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556ffc9-aeb6-411b-832a-f327a3e9c546 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:55:03.519714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a85552-46b3-490d-be3c-49b201045577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:55:03.520300 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a85552-46b3-490d-be3c-49b201045577 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:03.520575 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a85552-46b3-490d-be3c-49b201045577 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:03.520874 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a85552-46b3-490d-be3c-49b201045577 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:03.521589 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a85552-46b3-490d-be3c-49b201045577 None admin] RBAC: Adding query filter params (role.id=d8debbb193834fac9cc25f81726f3a67, scope.project.id=0ee65b9a8229495bbe97318f060d6342, user.id=cfbaaf32475a4c82a34948602d8f4e35) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:55:03.521894 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a85552-46b3-490d-be3c-49b201045577 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:55:03.523309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a85552-46b3-490d-be3c-49b201045577 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:55:03.529023 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 483/964] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Oct 20 05:55:03 2022] GET /identity/v3/role_assignments?user.id=cfbaaf32475a4c82a34948602d8f4e35&scope.project.id=0ee65b9a8229495bbe97318f060d6342&role.id=d8debbb193834fac9cc25f81726f3a67 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:55:05.625463 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c45a14-2d6d-4361-a2a6-6c6c28851e2e None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:05.625979 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c45a14-2d6d-4361-a2a6-6c6c28851e2e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:05.625979 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c45a14-2d6d-4361-a2a6-6c6c28851e2e None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:05.627278 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 482/965] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:55:05.632494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293892e8-6cfa-4c7b-9c1e-fed254d31689 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:05.632892 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293892e8-6cfa-4c7b-9c1e-fed254d31689 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:05.633077 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293892e8-6cfa-4c7b-9c1e-fed254d31689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:05.651968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-293892e8-6cfa-4c7b-9c1e-fed254d31689 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:55:05.653184 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-293892e8-6cfa-4c7b-9c1e-fed254d31689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:55:05.717588 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 484/966] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:55:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:55:05.992250 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c45a14-2d6d-4361-a2a6-6c6c28851e2e None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:55:06.028712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:55:06.096757 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c45a14-2d6d-4361-a2a6-6c6c28851e2e None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:55:06.129035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f14b1-e0b2-41c5-8c64-78eea1370eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:55:06.129651 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f14b1-e0b2-41c5-8c64-78eea1370eab None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:06.129928 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f14b1-e0b2-41c5-8c64-78eea1370eab None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:06.130198 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f14b1-e0b2-41c5-8c64-78eea1370eab None admin] PATH_INFO: `/v3/services/placement` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:06.130774 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f14b1-e0b2-41c5-8c64-78eea1370eab None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:55:06.132123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f14b1-e0b2-41c5-8c64-78eea1370eab None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:55:06.140883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-6b2f14b1-e0b2-41c5-8c64-78eea1370eab None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 05:55:06.140883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:55:06.140883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:55:06.140883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:55:06.140883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:55:06.140883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:55:06.140883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:55:06.140883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:55:06.140883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 05:55:06.140883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:55:06.140883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:55:06.140883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:55:06.140883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:55:06.140883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:55:06.140883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:55:06.140883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:55:06.140883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:55:06.142071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:55:06.142071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:55:06.142071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:55:06.142071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:55:06.142071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:55:06.142071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:55:06.142071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:55:06.142071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:55:06.142071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:55:06.142071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:55:06.142071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:55:06.142071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:55:06.142071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:55:06.142071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:55:06.142071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:55:06.142071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:55:06.142935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:55:06.142935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:55:06.142935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:55:06.142935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:55:06.142935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:55:06.142935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:55:06.142935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:55:06.142935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:55:06.142935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:55:06.142935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:55:06.142935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:55:06.142935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:55:06.142935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 05:55:06.142935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:55:06.142935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 483/967] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Oct 20 05:55:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:55:06.147987 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293892e8-6cfa-4c7b-9c1e-fed254d31689 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:55:06.234406 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:55:06.243672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293892e8-6cfa-4c7b-9c1e-fed254d31689 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:55:06.275523 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2457dad-846c-40c0-8d98-9a629421c637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:55:06.276626 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2457dad-846c-40c0-8d98-9a629421c637 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:06.276925 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2457dad-846c-40c0-8d98-9a629421c637 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:06.277192 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2457dad-846c-40c0-8d98-9a629421c637 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:06.277799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2457dad-846c-40c0-8d98-9a629421c637 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:55:06.278093 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2457dad-846c-40c0-8d98-9a629421c637 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:55:06.279437 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2457dad-846c-40c0-8d98-9a629421c637 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:55:06.284021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 485/968] 10.222.0.20 () {56 vars in 1116 bytes} [Thu Oct 20 05:55:06 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:55:06.289845 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f14b1-e0b2-41c5-8c64-78eea1370eab None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:55:06.379904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:55:06.386833 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f14b1-e0b2-41c5-8c64-78eea1370eab None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:55:06.419047 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ed95f-111d-4a95-8511-50a45b07ec54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:55:06.419642 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ed95f-111d-4a95-8511-50a45b07ec54 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:06.419917 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ed95f-111d-4a95-8511-50a45b07ec54 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:06.420184 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ed95f-111d-4a95-8511-50a45b07ec54 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:06.420820 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ed95f-111d-4a95-8511-50a45b07ec54 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:55:06.421117 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ed95f-111d-4a95-8511-50a45b07ec54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:55:06.422456 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ed95f-111d-4a95-8511-50a45b07ec54 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:55:06.426714 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 484/969] 10.222.0.20 () {56 vars in 1116 bytes} [Thu Oct 20 05:55:06 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:55:08.519605 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac2636-b2a3-4151-a705-cdee29d01026 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:08.520122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac2636-b2a3-4151-a705-cdee29d01026 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:08.520122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac2636-b2a3-4151-a705-cdee29d01026 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:08.521483 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 486/970] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:55:08.527540 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b2d4e-744a-4c14-9c9e-c232913c3888 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:08.527913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b2d4e-744a-4c14-9c9e-c232913c3888 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:08.528126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b2d4e-744a-4c14-9c9e-c232913c3888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:08.546550 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-8f1b2d4e-744a-4c14-9c9e-c232913c3888 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:55:08.547727 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8f1b2d4e-744a-4c14-9c9e-c232913c3888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:55:08.612041 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 485/971] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:55:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:55:08.882404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac2636-b2a3-4151-a705-cdee29d01026 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:55:08.987164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac2636-b2a3-4151-a705-cdee29d01026 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:55:09.021842 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31d268-edad-42f6-9689-0c887319fb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:55:09.022441 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31d268-edad-42f6-9689-0c887319fb10 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:09.022745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31d268-edad-42f6-9689-0c887319fb10 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:09.023015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31d268-edad-42f6-9689-0c887319fb10 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:09.023667 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31d268-edad-42f6-9689-0c887319fb10 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:55:09.025052 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31d268-edad-42f6-9689-0c887319fb10 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:55:09.039017 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 487/972] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Oct 20 05:55:08 2022] POST /identity/v3/services => generated 246 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:55:11.138543 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440d638-0229-409a-9f33-db3adb96b5fc None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:11.139080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440d638-0229-409a-9f33-db3adb96b5fc None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:11.139080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440d638-0229-409a-9f33-db3adb96b5fc None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:11.140397 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 486/973] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:55:11.145639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6a710-6e3c-4bf5-8ad4-ce84351e175d None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:11.146013 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6a710-6e3c-4bf5-8ad4-ce84351e175d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:11.146194 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6a710-6e3c-4bf5-8ad4-ce84351e175d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:11.164436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-53f6a710-6e3c-4bf5-8ad4-ce84351e175d None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:55:11.165657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-53f6a710-6e3c-4bf5-8ad4-ce84351e175d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:55:11.174628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:55:11.237427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 488/974] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:55:11 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:55:11.503810 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440d638-0229-409a-9f33-db3adb96b5fc None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:55:11.591476 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:55:11.601872 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440d638-0229-409a-9f33-db3adb96b5fc None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:55:11.635208 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd399feb-ebc7-412c-a387-18fecdd60820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:55:11.635814 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd399feb-ebc7-412c-a387-18fecdd60820 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:11.636095 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd399feb-ebc7-412c-a387-18fecdd60820 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:11.636367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd399feb-ebc7-412c-a387-18fecdd60820 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:11.636967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd399feb-ebc7-412c-a387-18fecdd60820 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:55:11.638327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd399feb-ebc7-412c-a387-18fecdd60820 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:55:11.646898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-cd399feb-ebc7-412c-a387-18fecdd60820 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 05:55:11.646898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:55:11.646898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:55:11.646898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:55:11.646898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:55:11.646898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:55:11.646898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:55:11.646898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:55:11.646898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 05:55:11.646898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:55:11.646898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:55:11.646898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:55:11.646898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:55:11.646898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:55:11.646898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:55:11.646898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:55:11.646898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:55:11.647874 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:55:11.647874 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:55:11.647874 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:55:11.647874 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:55:11.647874 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:55:11.647874 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:55:11.647874 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:55:11.647874 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:55:11.647874 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:55:11.647874 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:55:11.647874 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:55:11.647874 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:55:11.647874 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:55:11.647874 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:55:11.647874 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:55:11.647874 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:55:11.648998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:55:11.648998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:55:11.648998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:55:11.648998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:55:11.648998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:55:11.648998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:55:11.648998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:55:11.648998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:55:11.648998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:55:11.648998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:55:11.648998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:55:11.648998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:55:11.648998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 05:55:11.648998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:55:11.648998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 487/975] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Oct 20 05:55:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:55:11.654184 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6a710-6e3c-4bf5-8ad4-ce84351e175d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:55:11.749590 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6a710-6e3c-4bf5-8ad4-ce84351e175d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:55:11.781719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72976704-bb6e-44ef-a851-ebbfa3adb5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:55:11.782249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72976704-bb6e-44ef-a851-ebbfa3adb5cc None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:11.782531 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72976704-bb6e-44ef-a851-ebbfa3adb5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:11.782801 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72976704-bb6e-44ef-a851-ebbfa3adb5cc None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:11.783414 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72976704-bb6e-44ef-a851-ebbfa3adb5cc None admin] RBAC: Adding query filter params (name=placement) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:55:11.783713 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72976704-bb6e-44ef-a851-ebbfa3adb5cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:55:11.785097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72976704-bb6e-44ef-a851-ebbfa3adb5cc None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:55:11.789958 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 489/976] 10.222.0.20 () {56 vars in 1116 bytes} [Thu Oct 20 05:55:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:55:11.796687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd399feb-ebc7-412c-a387-18fecdd60820 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:55:11.894732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd399feb-ebc7-412c-a387-18fecdd60820 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:55:11.927484 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e09e6-dfe5-4810-9f88-bf22656cd13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:55:11.928066 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e09e6-dfe5-4810-9f88-bf22656cd13e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:11.928342 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e09e6-dfe5-4810-9f88-bf22656cd13e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:11.928612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e09e6-dfe5-4810-9f88-bf22656cd13e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:11.929323 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e09e6-dfe5-4810-9f88-bf22656cd13e None admin] RBAC: Adding query filter params (interface=public, service_id=cc7bd03f9f9e40c9ba65914e079a7e12, region_id=RegionOne) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:55:11.929628 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e09e6-dfe5-4810-9f88-bf22656cd13e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:55:11.930974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e09e6-dfe5-4810-9f88-bf22656cd13e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:55:11.936246 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 488/977] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Oct 20 05:55:11 2022] GET /identity/v3/endpoints?service_id=cc7bd03f9f9e40c9ba65914e079a7e12&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:55:11.941774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72976704-bb6e-44ef-a851-ebbfa3adb5cc None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:55:11.945578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:55:12.049030 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72976704-bb6e-44ef-a851-ebbfa3adb5cc None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:55:12.081909 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be679030-1166-4a67-ace2-40047ffe352a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:55:12.082504 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be679030-1166-4a67-ace2-40047ffe352a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:12.082781 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be679030-1166-4a67-ace2-40047ffe352a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:12.083051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be679030-1166-4a67-ace2-40047ffe352a None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:12.083641 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be679030-1166-4a67-ace2-40047ffe352a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:55:12.085021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be679030-1166-4a67-ace2-40047ffe352a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:55:12.091771 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 490/978] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Oct 20 05:55:11 2022] GET /identity/v3/services => generated 2191 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 05:55:14.223377 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567e9c5-e280-4b97-b577-0e125b705480 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:14.223934 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567e9c5-e280-4b97-b577-0e125b705480 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:14.223934 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567e9c5-e280-4b97-b577-0e125b705480 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:14.225285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 489/979] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:55:14.231458 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965702b-449e-448b-96fc-45db71e24f8b None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:14.231893 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965702b-449e-448b-96fc-45db71e24f8b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:14.232014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965702b-449e-448b-96fc-45db71e24f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:14.250408 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-a965702b-449e-448b-96fc-45db71e24f8b None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:55:14.251587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a965702b-449e-448b-96fc-45db71e24f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:55:14.312330 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:55:14.318178 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 491/980] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:55:14 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 05:55:14.587296 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567e9c5-e280-4b97-b577-0e125b705480 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:55:14.606230 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:55:14.688409 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567e9c5-e280-4b97-b577-0e125b705480 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:55:14.720493 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e484f355-b374-4c25-9039-c42793412d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:55:14.721138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484f355-b374-4c25-9039-c42793412d0d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:14.721423 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484f355-b374-4c25-9039-c42793412d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:14.721691 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484f355-b374-4c25-9039-c42793412d0d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:14.722262 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e484f355-b374-4c25-9039-c42793412d0d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:55:14.723609 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e484f355-b374-4c25-9039-c42793412d0d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:55:14.732716 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-e484f355-b374-4c25-9039-c42793412d0d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 05:55:14.732716 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:55:14.732716 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 05:55:14.732716 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 05:55:14.732716 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 05:55:14.732716 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 05:55:14.732716 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 05:55:14.732716 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:55:14.732716 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 05:55:14.732716 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:55:14.732716 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 05:55:14.732716 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:55:14.732716 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 05:55:14.732716 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 05:55:14.732716 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 05:55:14.732716 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 05:55:14.732716 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 05:55:14.733827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 05:55:14.733827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 05:55:14.733827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 05:55:14.733827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 05:55:14.733827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 05:55:14.733827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 05:55:14.733827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 05:55:14.733827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 05:55:14.733827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 05:55:14.733827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 05:55:14.733827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 05:55:14.733827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 05:55:14.733827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 05:55:14.733827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 05:55:14.733827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 05:55:14.733827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 05:55:14.734972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 05:55:14.734972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 05:55:14.734972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 05:55:14.734972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 05:55:14.734972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 05:55:14.734972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 05:55:14.734972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 05:55:14.734972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 05:55:14.734972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 05:55:14.734972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 05:55:14.734972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 05:55:14.734972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 05:55:14.734972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 05:55:14.734972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 05:55:14.734972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 490/981] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Oct 20 05:55:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 05:55:14.739698 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965702b-449e-448b-96fc-45db71e24f8b None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:55:14.834111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965702b-449e-448b-96fc-45db71e24f8b None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:55:14.866959 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5782a6d-a0d9-41e8-90b8-5237e72f994b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:55:14.867495 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5782a6d-a0d9-41e8-90b8-5237e72f994b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:14.867775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5782a6d-a0d9-41e8-90b8-5237e72f994b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:14.868046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5782a6d-a0d9-41e8-90b8-5237e72f994b None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:14.868659 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5782a6d-a0d9-41e8-90b8-5237e72f994b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 05:55:14.869548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5782a6d-a0d9-41e8-90b8-5237e72f994b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:55:14.871011 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5782a6d-a0d9-41e8-90b8-5237e72f994b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:55:14.875885 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 492/982] 10.222.0.20 () {56 vars in 1116 bytes} [Thu Oct 20 05:55:14 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 05:55:14.881498 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e484f355-b374-4c25-9039-c42793412d0d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:55:14.964750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:55:14.984101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e484f355-b374-4c25-9039-c42793412d0d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:55:15.022713 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277d493-5f93-4fa4-a169-117c2b15942f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:55:15.023319 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277d493-5f93-4fa4-a169-117c2b15942f None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:55:15.023592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277d493-5f93-4fa4-a169-117c2b15942f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:55:15.023859 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277d493-5f93-4fa4-a169-117c2b15942f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:55:15.024516 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c277d493-5f93-4fa4-a169-117c2b15942f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:55:15.026493 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c277d493-5f93-4fa4-a169-117c2b15942f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:55:15.045135 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 491/983] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 05:55:14 2022] POST /identity/v3/endpoints => generated 331 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 05:55:24.873118 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=69 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:55:25.041358 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:18.782070 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b031e39-e3e5-4a1f-a149-7a51cd912520 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:18.782553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b031e39-e3e5-4a1f-a149-7a51cd912520 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:18.782633 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b031e39-e3e5-4a1f-a149-7a51cd912520 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:18.787904 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 493/984] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:56:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:56:18.789340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e4ec0-8e0b-4b97-b037-8e22087973e3 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:18.789749 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e4ec0-8e0b-4b97-b037-8e22087973e3 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:18.789878 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e4ec0-8e0b-4b97-b037-8e22087973e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:18.911299 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-698e4ec0-8e0b-4b97-b037-8e22087973e3 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:56:18.913397 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-698e4ec0-8e0b-4b97-b037-8e22087973e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:19.043768 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 492/985] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:56:19.342985 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01baca5-6230-469c-a296-8f6b033efccf None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:19.343644 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01baca5-6230-469c-a296-8f6b033efccf None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:19.344125 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01baca5-6230-469c-a296-8f6b033efccf None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:19.345601 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 494/986] 10.222.0.20 () {54 vars in 964 bytes} [Thu Oct 20 05:56:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:56:19.352550 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054da8c-31ca-41a6-8fa7-49dd85860e63 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:19.352965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054da8c-31ca-41a6-8fa7-49dd85860e63 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:19.353313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054da8c-31ca-41a6-8fa7-49dd85860e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:19.445253 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-2054da8c-31ca-41a6-8fa7-49dd85860e63 None None] MFA Rules not processed for user `9072ce66e7174837aeb8c7898026fcce`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:56:19.446456 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2054da8c-31ca-41a6-8fa7-49dd85860e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:19.469940 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:19.531107 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 493/987] 10.222.0.20 () {58 vars in 1061 bytes} [Thu Oct 20 05:56:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:56:19.540834 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01baca5-6230-469c-a296-8f6b033efccf None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:56:19.542567 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e01baca5-6230-469c-a296-8f6b033efccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:19.661895 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01baca5-6230-469c-a296-8f6b033efccf None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:56:19.746918 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be77207-a0ec-4c78-859d-f3e99845df1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:56:19.747534 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be77207-a0ec-4c78-859d-f3e99845df1d service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:19.747813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be77207-a0ec-4c78-859d-f3e99845df1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:19.748083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be77207-a0ec-4c78-859d-f3e99845df1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:19.749722 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-4be77207-a0ec-4c78-859d-f3e99845df1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:19.808370 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:19.867020 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be77207-a0ec-4c78-859d-f3e99845df1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:56:19.868461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be77207-a0ec-4c78-859d-f3e99845df1d service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:56:19.888758 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 495/988] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:56:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:56:22.355452 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee2e8c-7b2c-45b0-b522-109d8cc252f5 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:22.356021 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee2e8c-7b2c-45b0-b522-109d8cc252f5 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:22.356021 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee2e8c-7b2c-45b0-b522-109d8cc252f5 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:22.357263 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 494/989] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:56:22.362411 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c31fd8-fc3b-4acd-b558-a2f669349600 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:22.362841 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c31fd8-fc3b-4acd-b558-a2f669349600 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:22.363060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c31fd8-fc3b-4acd-b558-a2f669349600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:22.445045 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f9c31fd8-fc3b-4acd-b558-a2f669349600 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:56:22.446257 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f9c31fd8-fc3b-4acd-b558-a2f669349600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:22.542416 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 496/990] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:56:22.843004 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03b3a4-7795-444a-ba1d-720fc595e8f1 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:22.843284 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03b3a4-7795-444a-ba1d-720fc595e8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:22.843555 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03b3a4-7795-444a-ba1d-720fc595e8f1 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:22.845010 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 495/991] 10.222.0.20 () {54 vars in 964 bytes} [Thu Oct 20 05:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:56:22.851698 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7085e-3c98-4cff-95b6-48279f439799 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:22.851990 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7085e-3c98-4cff-95b6-48279f439799 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:22.852267 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7085e-3c98-4cff-95b6-48279f439799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:22.864747 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:22.932207 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c3f7085e-3c98-4cff-95b6-48279f439799 None None] MFA Rules not processed for user `9072ce66e7174837aeb8c7898026fcce`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:56:22.933483 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c3f7085e-3c98-4cff-95b6-48279f439799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:23.011157 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 497/992] 10.222.0.20 () {58 vars in 1061 bytes} [Thu Oct 20 05:56:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:56:23.019884 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03b3a4-7795-444a-ba1d-720fc595e8f1 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:56:23.021492 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-dc03b3a4-7795-444a-ba1d-720fc595e8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:23.123313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03b3a4-7795-444a-ba1d-720fc595e8f1 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:56:23.196055 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:23.198320 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57772af7-b936-4bb5-aa58-fc254f9c74d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:56:23.199091 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57772af7-b936-4bb5-aa58-fc254f9c74d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:23.199583 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57772af7-b936-4bb5-aa58-fc254f9c74d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:23.200052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57772af7-b936-4bb5-aa58-fc254f9c74d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:23.201702 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-57772af7-b936-4bb5-aa58-fc254f9c74d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:23.307862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57772af7-b936-4bb5-aa58-fc254f9c74d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:56:23.309344 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57772af7-b936-4bb5-aa58-fc254f9c74d0 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:56:23.327828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 496/993] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:56:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:56:25.995800 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edf97b-21ef-4fa3-8881-cb866910e3be None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:25.996363 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edf97b-21ef-4fa3-8881-cb866910e3be None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:25.996363 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edf97b-21ef-4fa3-8881-cb866910e3be None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:25.997697 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 498/994] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:56:26.002830 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ce74c-f5d9-4184-a672-b2919eaccc60 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:26.003253 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ce74c-f5d9-4184-a672-b2919eaccc60 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:26.003457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ce74c-f5d9-4184-a672-b2919eaccc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:26.067563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-6d3ce74c-f5d9-4184-a672-b2919eaccc60 None None] MFA Rules not processed for user `3ca0dc0787674530a74f83cfa41df3d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:56:26.068809 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6d3ce74c-f5d9-4184-a672-b2919eaccc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:26.122389 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:26.166915 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 497/995] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:56:26 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:56:26.451170 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42edf97b-21ef-4fa3-8881-cb866910e3be None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:56:26.452785 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-42edf97b-21ef-4fa3-8881-cb866910e3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:26.556913 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42edf97b-21ef-4fa3-8881-cb866910e3be None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:56:26.559902 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:26.631008 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff7e640-383a-4e25-aa75-836ff1b8b57b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:56:26.632140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7e640-383a-4e25-aa75-836ff1b8b57b service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:26.632415 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7e640-383a-4e25-aa75-836ff1b8b57b service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:26.632684 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7e640-383a-4e25-aa75-836ff1b8b57b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:26.634308 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2ff7e640-383a-4e25-aa75-836ff1b8b57b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:26.741784 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff7e640-383a-4e25-aa75-836ff1b8b57b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:56:26.743221 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff7e640-383a-4e25-aa75-836ff1b8b57b service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:56:26.761749 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 499/996] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:56:26 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:56:29.410117 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d32d64-7bf0-4f9a-a47d-4412b3c0be26 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:29.410643 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d32d64-7bf0-4f9a-a47d-4412b3c0be26 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:29.410643 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d32d64-7bf0-4f9a-a47d-4412b3c0be26 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:29.412003 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 498/997] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:56:29.417310 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf274cd-90a5-427e-a249-6dabb88418cb None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:29.417688 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf274cd-90a5-427e-a249-6dabb88418cb None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:29.417856 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf274cd-90a5-427e-a249-6dabb88418cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:29.458870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:29.483417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-dbf274cd-90a5-427e-a249-6dabb88418cb None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:56:29.484598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-dbf274cd-90a5-427e-a249-6dabb88418cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:29.577406 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 500/998] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:56:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:56:29.856368 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d32d64-7bf0-4f9a-a47d-4412b3c0be26 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:56:29.857960 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e6d32d64-7bf0-4f9a-a47d-4412b3c0be26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:29.960865 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d32d64-7bf0-4f9a-a47d-4412b3c0be26 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:56:30.032416 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0efb0-283c-485f-9c5f-173bf703d861 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:56:30.033031 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0efb0-283c-485f-9c5f-173bf703d861 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:30.033305 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0efb0-283c-485f-9c5f-173bf703d861 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:30.033571 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0efb0-283c-485f-9c5f-173bf703d861 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:30.035143 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-aff0efb0-283c-485f-9c5f-173bf703d861 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:30.042377 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:30.142175 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff0efb0-283c-485f-9c5f-173bf703d861 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:56:30.143612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff0efb0-283c-485f-9c5f-173bf703d861 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:56:30.161911 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 499/999] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:56:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:56:32.926134 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59212010-05e2-4282-8cc9-693853e69dec None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:32.926692 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59212010-05e2-4282-8cc9-693853e69dec None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:32.926692 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59212010-05e2-4282-8cc9-693853e69dec None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:32.928203 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 501/1000] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:56:32.933210 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef04d43-fe5f-4734-a6bb-de95c19fc05e None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:32.933634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef04d43-fe5f-4734-a6bb-de95c19fc05e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:32.933736 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef04d43-fe5f-4734-a6bb-de95c19fc05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:33.005164 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-5ef04d43-fe5f-4734-a6bb-de95c19fc05e None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:56:33.007019 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5ef04d43-fe5f-4734-a6bb-de95c19fc05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:33.069881 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:33.113468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 500/1001] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:56:33.398611 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59212010-05e2-4282-8cc9-693853e69dec None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:56:33.400134 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-59212010-05e2-4282-8cc9-693853e69dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:33.504313 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59212010-05e2-4282-8cc9-693853e69dec None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:56:33.517937 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:33.580734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce86415-cf66-4d3f-9267-819909e35ee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:56:33.581250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce86415-cf66-4d3f-9267-819909e35ee5 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:33.581533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce86415-cf66-4d3f-9267-819909e35ee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:33.581804 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce86415-cf66-4d3f-9267-819909e35ee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:33.583394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0ce86415-cf66-4d3f-9267-819909e35ee5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:33.691927 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce86415-cf66-4d3f-9267-819909e35ee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:56:33.693474 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce86415-cf66-4d3f-9267-819909e35ee5 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:56:33.711451 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 502/1002] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:56:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:56:35.930278 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30955ed7-3ef8-403b-946f-9562c3a79785 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:35.930832 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30955ed7-3ef8-403b-946f-9562c3a79785 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:35.930832 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30955ed7-3ef8-403b-946f-9562c3a79785 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:35.932166 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 501/1003] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:56:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:56:35.937363 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118856a8-2bdf-4935-a371-cdf3ea57a563 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:35.937780 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118856a8-2bdf-4935-a371-cdf3ea57a563 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:35.937922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118856a8-2bdf-4935-a371-cdf3ea57a563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:35.987801 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:36.007002 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-118856a8-2bdf-4935-a371-cdf3ea57a563 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:56:36.008213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-118856a8-2bdf-4935-a371-cdf3ea57a563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:36.113201 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 503/1004] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:56:36.395745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30955ed7-3ef8-403b-946f-9562c3a79785 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:56:36.398004 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-30955ed7-3ef8-403b-946f-9562c3a79785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:36.499779 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30955ed7-3ef8-403b-946f-9562c3a79785 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:56:36.573145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1663e3b-ab0e-4f0c-9c93-f0cb957a1c1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:56:36.573749 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1663e3b-ab0e-4f0c-9c93-f0cb957a1c1e service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:36.574028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1663e3b-ab0e-4f0c-9c93-f0cb957a1c1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:36.574298 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1663e3b-ab0e-4f0c-9c93-f0cb957a1c1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:36.575884 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a1663e3b-ab0e-4f0c-9c93-f0cb957a1c1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:36.593138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:36.684340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1663e3b-ab0e-4f0c-9c93-f0cb957a1c1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:56:36.685733 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1663e3b-ab0e-4f0c-9c93-f0cb957a1c1e service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:56:36.704440 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 502/1005] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:56:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:56:38.902442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dff536-acf8-4149-bb07-3f49c8254870 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:38.902978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dff536-acf8-4149-bb07-3f49c8254870 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:38.902978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dff536-acf8-4149-bb07-3f49c8254870 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:38.904410 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 504/1006] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:56:38.909612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca511a0-be47-4c92-bb35-d6e890bc247b None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:38.910022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca511a0-be47-4c92-bb35-d6e890bc247b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:38.910159 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca511a0-be47-4c92-bb35-d6e890bc247b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:38.976921 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-cca511a0-be47-4c92-bb35-d6e890bc247b None None] MFA Rules not processed for user `3ca0dc0787674530a74f83cfa41df3d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:56:38.978105 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-cca511a0-be47-4c92-bb35-d6e890bc247b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:39.043977 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:39.076523 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 503/1007] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:56:38 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:56:39.371320 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dff536-acf8-4149-bb07-3f49c8254870 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:56:39.372968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-13dff536-acf8-4149-bb07-3f49c8254870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:39.481533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dff536-acf8-4149-bb07-3f49c8254870 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:56:39.503663 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:39.560801 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9637d-e175-41f5-8cdc-864ddc7c799f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:56:39.561416 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9637d-e175-41f5-8cdc-864ddc7c799f service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:39.561695 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9637d-e175-41f5-8cdc-864ddc7c799f service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:39.561968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9637d-e175-41f5-8cdc-864ddc7c799f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:39.563550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d8e9637d-e175-41f5-8cdc-864ddc7c799f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:39.671804 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9637d-e175-41f5-8cdc-864ddc7c799f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:56:39.673304 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9637d-e175-41f5-8cdc-864ddc7c799f service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:56:39.691395 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 505/1008] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:56:39 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:56:42.257060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf203b9-7e89-4692-9b31-bf537cee6d22 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:42.257596 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf203b9-7e89-4692-9b31-bf537cee6d22 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:42.257596 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf203b9-7e89-4692-9b31-bf537cee6d22 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:42.259695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 504/1009] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:56:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:56:42.265274 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ef8ee-a534-48a5-91ee-0a4fbc692780 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:42.265698 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ef8ee-a534-48a5-91ee-0a4fbc692780 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:42.265825 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ef8ee-a534-48a5-91ee-0a4fbc692780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:42.329633 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:42.332282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-9e8ef8ee-a534-48a5-91ee-0a4fbc692780 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:56:42.334166 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-9e8ef8ee-a534-48a5-91ee-0a4fbc692780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:42.429835 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 506/1010] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:56:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:56:42.712676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf203b9-7e89-4692-9b31-bf537cee6d22 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:56:42.714264 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-bcf203b9-7e89-4692-9b31-bf537cee6d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:42.816071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf203b9-7e89-4692-9b31-bf537cee6d22 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:56:42.887242 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb572d3-ed1a-45b2-bab3-96467339fe35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:56:42.887833 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb572d3-ed1a-45b2-bab3-96467339fe35 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:42.888107 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb572d3-ed1a-45b2-bab3-96467339fe35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:42.888374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb572d3-ed1a-45b2-bab3-96467339fe35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:42.889965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-dbb572d3-ed1a-45b2-bab3-96467339fe35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:42.916854 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:42.996181 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb572d3-ed1a-45b2-bab3-96467339fe35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:56:42.997554 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb572d3-ed1a-45b2-bab3-96467339fe35 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:56:43.015308 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 505/1011] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:56:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:56:45.190872 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f50945-18de-4a55-888a-ebb8d186c571 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:45.191405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f50945-18de-4a55-888a-ebb8d186c571 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:45.191405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f50945-18de-4a55-888a-ebb8d186c571 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:45.193566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 507/1012] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:56:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:56:45.199022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c5b41-7d9c-4645-839a-77b851dae052 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:45.199423 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c5b41-7d9c-4645-839a-77b851dae052 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:45.199571 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c5b41-7d9c-4645-839a-77b851dae052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:45.274818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-6d1c5b41-7d9c-4645-839a-77b851dae052 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:56:45.276101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6d1c5b41-7d9c-4645-839a-77b851dae052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:45.364597 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:45.390742 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 506/1013] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:56:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:56:45.720144 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f50945-18de-4a55-888a-ebb8d186c571 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:56:45.721697 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c8f50945-18de-4a55-888a-ebb8d186c571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:45.840961 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f50945-18de-4a55-888a-ebb8d186c571 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:56:45.873026 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:45.920196 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5075bf-7728-45c1-9383-13d257ba98cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:56:45.920839 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5075bf-7728-45c1-9383-13d257ba98cb service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:45.921121 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5075bf-7728-45c1-9383-13d257ba98cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:45.921402 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5075bf-7728-45c1-9383-13d257ba98cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:45.923004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0f5075bf-7728-45c1-9383-13d257ba98cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:46.031201 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5075bf-7728-45c1-9383-13d257ba98cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:56:46.032638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5075bf-7728-45c1-9383-13d257ba98cb service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:56:46.050935 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 508/1014] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:56:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:56:48.667796 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78400d9c-915e-4d0b-b14e-fc25ff05754f None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:48.668350 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78400d9c-915e-4d0b-b14e-fc25ff05754f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:48.668350 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78400d9c-915e-4d0b-b14e-fc25ff05754f None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:48.669727 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 507/1015] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:56:48.674921 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43536b-5308-4ddb-a76a-f2d6878dbbff None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:48.675359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43536b-5308-4ddb-a76a-f2d6878dbbff None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:48.675455 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43536b-5308-4ddb-a76a-f2d6878dbbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:48.741467 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-6a43536b-5308-4ddb-a76a-f2d6878dbbff None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:56:48.742672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6a43536b-5308-4ddb-a76a-f2d6878dbbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:48.750885 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:48.839436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 509/1016] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:56:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:56:49.132964 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78400d9c-915e-4d0b-b14e-fc25ff05754f None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:56:49.134496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-78400d9c-915e-4d0b-b14e-fc25ff05754f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:49.234510 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78400d9c-915e-4d0b-b14e-fc25ff05754f None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:56:49.310686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cb9a6-18c9-432f-98ca-cfa4806c9c46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:56:49.311292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cb9a6-18c9-432f-98ca-cfa4806c9c46 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:49.311569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cb9a6-18c9-432f-98ca-cfa4806c9c46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:49.311839 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cb9a6-18c9-432f-98ca-cfa4806c9c46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:49.313470 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8e0cb9a6-18c9-432f-98ca-cfa4806c9c46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:49.350814 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:49.424242 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0cb9a6-18c9-432f-98ca-cfa4806c9c46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:56:49.425704 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0cb9a6-18c9-432f-98ca-cfa4806c9c46 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:56:49.444651 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 508/1017] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:56:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:56:53.572923 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dae02d-6ac7-49a1-a914-47b94e13bd22 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:53.573467 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dae02d-6ac7-49a1-a914-47b94e13bd22 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:53.573467 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dae02d-6ac7-49a1-a914-47b94e13bd22 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:53.574706 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 510/1018] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:56:53.580140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707fa996-385e-47aa-8023-4fc0bda1de53 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:53.580587 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707fa996-385e-47aa-8023-4fc0bda1de53 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:53.580676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707fa996-385e-47aa-8023-4fc0bda1de53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:53.648455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-707fa996-385e-47aa-8023-4fc0bda1de53 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:56:53.649692 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-707fa996-385e-47aa-8023-4fc0bda1de53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:53.733881 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:53.745660 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 509/1019] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:56:54.029670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dae02d-6ac7-49a1-a914-47b94e13bd22 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:56:54.031202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e3dae02d-6ac7-49a1-a914-47b94e13bd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:54.137577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dae02d-6ac7-49a1-a914-47b94e13bd22 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:56:54.176233 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:54.212946 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e9724-0ab7-487e-8e14-6f38bfc59e40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:56:54.213552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e9724-0ab7-487e-8e14-6f38bfc59e40 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:54.213826 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e9724-0ab7-487e-8e14-6f38bfc59e40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:54.214094 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e9724-0ab7-487e-8e14-6f38bfc59e40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:54.215655 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c26e9724-0ab7-487e-8e14-6f38bfc59e40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:54.329154 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26e9724-0ab7-487e-8e14-6f38bfc59e40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:56:54.330599 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26e9724-0ab7-487e-8e14-6f38bfc59e40 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:56:54.349547 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 511/1020] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:56:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:56:57.123574 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15715db2-47b5-4364-bd1d-c18dfc7a81d1 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:57.124093 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15715db2-47b5-4364-bd1d-c18dfc7a81d1 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:57.124093 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15715db2-47b5-4364-bd1d-c18dfc7a81d1 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:57.125388 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 510/1021] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:56:57.130837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19130634-4fb1-4279-82f3-61edb9037cb1 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:57.131298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19130634-4fb1-4279-82f3-61edb9037cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:57.131473 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19130634-4fb1-4279-82f3-61edb9037cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:57.198863 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-19130634-4fb1-4279-82f3-61edb9037cb1 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:56:57.200066 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-19130634-4fb1-4279-82f3-61edb9037cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:57.219463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:57.302513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 512/1022] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:56:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:56:57.587261 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15715db2-47b5-4364-bd1d-c18dfc7a81d1 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:56:57.588872 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-15715db2-47b5-4364-bd1d-c18dfc7a81d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:57.696921 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15715db2-47b5-4364-bd1d-c18dfc7a81d1 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:56:57.776194 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bff45e-055d-42bc-9dd4-c5228bb3ea52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:56:57.776831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bff45e-055d-42bc-9dd4-c5228bb3ea52 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:56:57.777108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bff45e-055d-42bc-9dd4-c5228bb3ea52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:56:57.777378 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bff45e-055d-42bc-9dd4-c5228bb3ea52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:56:57.778955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-83bff45e-055d-42bc-9dd4-c5228bb3ea52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:56:57.821751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:56:57.888373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bff45e-055d-42bc-9dd4-c5228bb3ea52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:56:57.889860 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bff45e-055d-42bc-9dd4-c5228bb3ea52 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:56:57.908725 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 511/1023] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:56:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:57:02.099227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f592757-9d51-4075-baef-0366bb841f11 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:02.099737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f592757-9d51-4075-baef-0366bb841f11 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:02.099737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f592757-9d51-4075-baef-0366bb841f11 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:02.101786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 513/1024] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:57:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:57:02.106910 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02649c-b01a-4551-ace7-a8ec5f0f75be None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:02.107312 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02649c-b01a-4551-ace7-a8ec5f0f75be None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:02.107461 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02649c-b01a-4551-ace7-a8ec5f0f75be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:02.180630 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-3a02649c-b01a-4551-ace7-a8ec5f0f75be None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:57:02.182109 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3a02649c-b01a-4551-ace7-a8ec5f0f75be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:02.305247 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:02.308054 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 512/1025] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:57:02.588056 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f592757-9d51-4075-baef-0366bb841f11 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:57:02.589689 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5f592757-9d51-4075-baef-0366bb841f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:02.693729 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f592757-9d51-4075-baef-0366bb841f11 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:57:02.740755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:02.766275 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b5052-c577-4a5a-ae0f-b76daf138094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:57:02.766877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b5052-c577-4a5a-ae0f-b76daf138094 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:02.767165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b5052-c577-4a5a-ae0f-b76daf138094 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:02.767415 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b5052-c577-4a5a-ae0f-b76daf138094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:02.769014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7b1b5052-c577-4a5a-ae0f-b76daf138094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:02.874711 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b5052-c577-4a5a-ae0f-b76daf138094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:57:02.876051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b5052-c577-4a5a-ae0f-b76daf138094 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:57:02.894335 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 514/1026] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:57:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:57:07.783718 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5b216-1883-4145-aa50-5e8012d3b0e6 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:07.784148 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5b216-1883-4145-aa50-5e8012d3b0e6 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:07.784277 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5b216-1883-4145-aa50-5e8012d3b0e6 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:07.787428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 513/1027] 10.222.0.20 () {54 vars in 921 bytes} [Thu Oct 20 05:57:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:57:07.792937 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68b0d7-cda3-479b-99be-75f5d251cfd3 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:07.793394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68b0d7-cda3-479b-99be-75f5d251cfd3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:07.793475 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68b0d7-cda3-479b-99be-75f5d251cfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:07.897789 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-5a68b0d7-cda3-479b-99be-75f5d251cfd3 None None] MFA Rules not processed for user `cfbaaf32475a4c82a34948602d8f4e35`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:57:07.899287 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5a68b0d7-cda3-479b-99be-75f5d251cfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:07.945396 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:08.054779 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 515/1028] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Oct 20 05:57:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:57:12.404590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794ca71-7288-4d8d-9c84-36a47086a423 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:12.405165 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794ca71-7288-4d8d-9c84-36a47086a423 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:12.405165 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794ca71-7288-4d8d-9c84-36a47086a423 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:12.406649 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 514/1029] 10.222.0.20 () {54 vars in 921 bytes} [Thu Oct 20 05:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:57:12.412541 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf11f8-3d87-41b8-8a38-9f976395bc9c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:12.412967 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf11f8-3d87-41b8-8a38-9f976395bc9c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:12.413194 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf11f8-3d87-41b8-8a38-9f976395bc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:12.496999 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-82cf11f8-3d87-41b8-8a38-9f976395bc9c None None] MFA Rules not processed for user `cfbaaf32475a4c82a34948602d8f4e35`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:57:12.498240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-82cf11f8-3d87-41b8-8a38-9f976395bc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:12.577187 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:12.622592 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 516/1030] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Oct 20 05:57:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 05:57:13.914627 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a71057-80fd-44f9-951c-35ad8ee01979 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:13.915152 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a71057-80fd-44f9-951c-35ad8ee01979 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:13.915152 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a71057-80fd-44f9-951c-35ad8ee01979 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:13.916474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 515/1031] 10.222.0.20 () {54 vars in 921 bytes} [Thu Oct 20 05:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:57:13.921365 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76540910-6f37-44ab-8cd7-374d0800b978 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:13.921801 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76540910-6f37-44ab-8cd7-374d0800b978 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:13.921898 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76540910-6f37-44ab-8cd7-374d0800b978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:13.997836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-76540910-6f37-44ab-8cd7-374d0800b978 None None] MFA Rules not processed for user `cfbaaf32475a4c82a34948602d8f4e35`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:57:13.999023 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-76540910-6f37-44ab-8cd7-374d0800b978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:14.106098 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 517/1032] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Oct 20 05:57:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 05:57:17.624647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5baf20-069a-4f69-bda1-60aef904ffc9 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:17.625184 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5baf20-069a-4f69-bda1-60aef904ffc9 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:17.625184 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5baf20-069a-4f69-bda1-60aef904ffc9 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:17.626502 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 516/1033] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:57:17.632744 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0983a-5cd1-40e3-8357-1c37dde7a857 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:17.633163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0983a-5cd1-40e3-8357-1c37dde7a857 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:17.633351 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0983a-5cd1-40e3-8357-1c37dde7a857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:17.672023 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:17.709043 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-69c0983a-5cd1-40e3-8357-1c37dde7a857 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:57:17.710788 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-69c0983a-5cd1-40e3-8357-1c37dde7a857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:17.819718 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 518/1034] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:57:18.123230 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7ab3d-cbf2-4dda-aab3-314e53361492 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:18.123517 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7ab3d-cbf2-4dda-aab3-314e53361492 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:18.123787 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7ab3d-cbf2-4dda-aab3-314e53361492 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:18.125343 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 517/1035] 10.222.0.20 () {54 vars in 961 bytes} [Thu Oct 20 05:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:57:18.149409 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8956f9-f63b-4806-9782-5bf8a1e2cb4a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:18.149938 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8956f9-f63b-4806-9782-5bf8a1e2cb4a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:18.150388 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8956f9-f63b-4806-9782-5bf8a1e2cb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:18.233745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-6a8956f9-f63b-4806-9782-5bf8a1e2cb4a None None] MFA Rules not processed for user `ae2c1c886cd94d9f90082f66e7e8c9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:57:18.234972 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6a8956f9-f63b-4806-9782-5bf8a1e2cb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:18.290610 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:18.372017 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 519/1036] 10.222.0.20 () {58 vars in 1058 bytes} [Thu Oct 20 05:57:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:57:18.380614 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e7ab3d-cbf2-4dda-aab3-314e53361492 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:57:18.382151 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b0e7ab3d-cbf2-4dda-aab3-314e53361492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:18.537837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e7ab3d-cbf2-4dda-aab3-314e53361492 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:57:18.615164 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:18.660213 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907ff16-4179-47d9-8975-deafd4635077 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:57:18.660887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907ff16-4179-47d9-8975-deafd4635077 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:18.661146 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907ff16-4179-47d9-8975-deafd4635077 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:18.661405 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907ff16-4179-47d9-8975-deafd4635077 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:18.662991 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f907ff16-4179-47d9-8975-deafd4635077 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:18.762172 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907ff16-4179-47d9-8975-deafd4635077 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:57:18.763770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907ff16-4179-47d9-8975-deafd4635077 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:57:18.783337 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 518/1037] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:57:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:57:21.179634 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13525938-cfb1-43c9-a043-7d5f2fbeeac3 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:21.179634 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13525938-cfb1-43c9-a043-7d5f2fbeeac3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:21.179634 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13525938-cfb1-43c9-a043-7d5f2fbeeac3 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:21.179634 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 520/1038] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:57:21.184633 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a6e08-d645-4ffe-9cb5-0b463e7adc4e None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:21.185227 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a6e08-d645-4ffe-9cb5-0b463e7adc4e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:21.185227 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a6e08-d645-4ffe-9cb5-0b463e7adc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:21.257038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-117a6e08-d645-4ffe-9cb5-0b463e7adc4e None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:57:21.258256 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-117a6e08-d645-4ffe-9cb5-0b463e7adc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:21.300265 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:21.370927 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 519/1039] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:57:21.657435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13525938-cfb1-43c9-a043-7d5f2fbeeac3 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:57:21.659362 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-13525938-cfb1-43c9-a043-7d5f2fbeeac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:21.834228 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13525938-cfb1-43c9-a043-7d5f2fbeeac3 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:57:21.845918 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:21.980847 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4cccc-a727-4e2b-8a1d-dce2001d6b98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:57:21.981451 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4cccc-a727-4e2b-8a1d-dce2001d6b98 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:21.981721 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4cccc-a727-4e2b-8a1d-dce2001d6b98 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:21.982269 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4cccc-a727-4e2b-8a1d-dce2001d6b98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:21.983572 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8fe4cccc-a727-4e2b-8a1d-dce2001d6b98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:22.079673 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4cccc-a727-4e2b-8a1d-dce2001d6b98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:57:22.081792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4cccc-a727-4e2b-8a1d-dce2001d6b98 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:57:22.100299 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 521/1040] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:57:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:57:24.594859 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b4d77-cafc-493c-89b8-f33c280752b0 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:24.595491 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b4d77-cafc-493c-89b8-f33c280752b0 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:24.595491 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b4d77-cafc-493c-89b8-f33c280752b0 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:24.597381 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 520/1041] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:57:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:57:24.602804 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f50663-fd03-405c-bb20-94df2394e73e None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:24.603250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f50663-fd03-405c-bb20-94df2394e73e None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:24.603345 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f50663-fd03-405c-bb20-94df2394e73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:24.618904 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:24.671983 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-d9f50663-fd03-405c-bb20-94df2394e73e None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:57:24.673211 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d9f50663-fd03-405c-bb20-94df2394e73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:24.774700 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 522/1042] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 05:57:25.067837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2531f1f-3717-4093-b842-b676d72e2ece None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:25.068115 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2531f1f-3717-4093-b842-b676d72e2ece None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:25.068572 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2531f1f-3717-4093-b842-b676d72e2ece None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:25.069907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 521/1043] 10.222.0.20 () {54 vars in 961 bytes} [Thu Oct 20 05:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:57:25.075495 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1636a-8a87-422b-9e1e-25d19a0b4311 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:25.075992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1636a-8a87-422b-9e1e-25d19a0b4311 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:25.076330 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1636a-8a87-422b-9e1e-25d19a0b4311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:25.155176 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f2b1636a-8a87-422b-9e1e-25d19a0b4311 None None] MFA Rules not processed for user `ae2c1c886cd94d9f90082f66e7e8c9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:57:25.156357 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f2b1636a-8a87-422b-9e1e-25d19a0b4311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:25.193133 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:25.279263 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 523/1044] 10.222.0.20 () {58 vars in 1058 bytes} [Thu Oct 20 05:57:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:57:25.286934 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2531f1f-3717-4093-b842-b676d72e2ece None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:57:25.288430 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e2531f1f-3717-4093-b842-b676d72e2ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:25.456205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2531f1f-3717-4093-b842-b676d72e2ece None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:57:25.479621 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:25.579816 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2101e8c-d95b-4b4c-9f0d-03374c68bf1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:57:25.580427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2101e8c-d95b-4b4c-9f0d-03374c68bf1f service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:25.581235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2101e8c-d95b-4b4c-9f0d-03374c68bf1f service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:25.581505 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2101e8c-d95b-4b4c-9f0d-03374c68bf1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:25.583205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d2101e8c-d95b-4b4c-9f0d-03374c68bf1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:25.678303 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2101e8c-d95b-4b4c-9f0d-03374c68bf1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:57:25.679933 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2101e8c-d95b-4b4c-9f0d-03374c68bf1f service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:57:25.701910 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 522/1045] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:57:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:57:25.803576 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aea8db-2c60-4bca-b4d7-7bb1c87abac3 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:25.804151 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aea8db-2c60-4bca-b4d7-7bb1c87abac3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:25.804237 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aea8db-2c60-4bca-b4d7-7bb1c87abac3 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:25.805529 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 524/1046] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:57:25.814555 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1d56a-0bec-43ab-84b6-b9592a755877 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:25.814847 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1d56a-0bec-43ab-84b6-b9592a755877 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:25.815177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1d56a-0bec-43ab-84b6-b9592a755877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:25.851746 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:25.897936 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d0a1d56a-0bec-43ab-84b6-b9592a755877 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:57:25.899475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d0a1d56a-0bec-43ab-84b6-b9592a755877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:26.025299 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 523/1047] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:57:26.364668 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150a5f7-a727-48d0-8d5d-4909bd5c80f7 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:26.364668 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150a5f7-a727-48d0-8d5d-4909bd5c80f7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:26.364668 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150a5f7-a727-48d0-8d5d-4909bd5c80f7 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:26.367306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 525/1048] 10.222.0.20 () {54 vars in 963 bytes} [Thu Oct 20 05:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:57:26.371129 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727974f5-e99d-4436-b976-f7a224cd6c0a None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:26.371415 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727974f5-e99d-4436-b976-f7a224cd6c0a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:26.371687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727974f5-e99d-4436-b976-f7a224cd6c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:26.454250 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-727974f5-e99d-4436-b976-f7a224cd6c0a None None] MFA Rules not processed for user `cfb00a74a99e47c1ac938f07d26a4bc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:57:26.455672 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-727974f5-e99d-4436-b976-f7a224cd6c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:26.510512 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:26.542983 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 524/1049] 10.222.0.20 () {58 vars in 1060 bytes} [Thu Oct 20 05:57:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:57:26.551537 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150a5f7-a727-48d0-8d5d-4909bd5c80f7 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:57:26.553205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0150a5f7-a727-48d0-8d5d-4909bd5c80f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:26.669067 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150a5f7-a727-48d0-8d5d-4909bd5c80f7 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:57:26.717575 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:26.762206 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e78db-9afa-4595-be4a-be6ae7bfd120 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:57:26.763197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e78db-9afa-4595-be4a-be6ae7bfd120 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:26.763672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e78db-9afa-4595-be4a-be6ae7bfd120 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:26.764144 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e78db-9afa-4595-be4a-be6ae7bfd120 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:26.766437 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e02e78db-9afa-4595-be4a-be6ae7bfd120 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:26.954464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e78db-9afa-4595-be4a-be6ae7bfd120 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:57:26.956676 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e78db-9afa-4595-be4a-be6ae7bfd120 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:57:26.989207 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 526/1050] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:57:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:57:27.963351 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0e418-f380-4658-9387-99acff861269 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:27.963864 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0e418-f380-4658-9387-99acff861269 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:27.963864 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0e418-f380-4658-9387-99acff861269 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:27.965248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 525/1051] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:57:27.970547 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2c5c9-6572-470c-9991-b39a628440b3 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:27.970943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2c5c9-6572-470c-9991-b39a628440b3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:27.971153 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2c5c9-6572-470c-9991-b39a628440b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:28.037533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-8ae2c5c9-6572-470c-9991-b39a628440b3 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:57:28.038750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8ae2c5c9-6572-470c-9991-b39a628440b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:28.060239 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:28.142600 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 527/1052] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:57:28.434775 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0e418-f380-4658-9387-99acff861269 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:57:28.436302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-94b0e418-f380-4658-9387-99acff861269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:28.599428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0e418-f380-4658-9387-99acff861269 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:57:28.656511 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:28.720201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba2fc9-6dc2-4e1d-9aff-dd63fbfed098 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:57:28.720842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba2fc9-6dc2-4e1d-9aff-dd63fbfed098 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:28.721789 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba2fc9-6dc2-4e1d-9aff-dd63fbfed098 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:28.722076 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba2fc9-6dc2-4e1d-9aff-dd63fbfed098 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:28.723662 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9eba2fc9-6dc2-4e1d-9aff-dd63fbfed098 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:28.815480 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba2fc9-6dc2-4e1d-9aff-dd63fbfed098 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:57:28.817256 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba2fc9-6dc2-4e1d-9aff-dd63fbfed098 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:57:28.835759 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 526/1053] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:57:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:57:31.070390 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e7ffd-efba-4623-b0ef-bfb5e91d93bc None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:31.070931 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e7ffd-efba-4623-b0ef-bfb5e91d93bc None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:31.070931 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e7ffd-efba-4623-b0ef-bfb5e91d93bc None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:31.072423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 528/1054] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:57:31.077629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da216a-4baf-465c-85a8-d382149d0cd2 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:31.078081 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da216a-4baf-465c-85a8-d382149d0cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:31.078166 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da216a-4baf-465c-85a8-d382149d0cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:31.146942 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-74da216a-4baf-465c-85a8-d382149d0cd2 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:57:31.148138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-74da216a-4baf-465c-85a8-d382149d0cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:31.155366 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:31.251384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 527/1055] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:57:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:57:31.545462 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e7ffd-efba-4623-b0ef-bfb5e91d93bc None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:57:31.547213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-494e7ffd-efba-4623-b0ef-bfb5e91d93bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:31.701330 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e7ffd-efba-4623-b0ef-bfb5e91d93bc None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:57:31.705074 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:31.819224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b0688-5e1a-4a10-87d3-66bc9637fc1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:57:31.819755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b0688-5e1a-4a10-87d3-66bc9637fc1b service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:31.820031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b0688-5e1a-4a10-87d3-66bc9637fc1b service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:31.820299 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b0688-5e1a-4a10-87d3-66bc9637fc1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:31.821899 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-708b0688-5e1a-4a10-87d3-66bc9637fc1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:31.915683 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b0688-5e1a-4a10-87d3-66bc9637fc1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:57:31.917295 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b0688-5e1a-4a10-87d3-66bc9637fc1b service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:57:31.935984 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 529/1056] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:57:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:57:34.153321 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f921a-f8d4-4b83-8d9a-a960c4f4fa14 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:34.153831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f921a-f8d4-4b83-8d9a-a960c4f4fa14 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:34.153831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f921a-f8d4-4b83-8d9a-a960c4f4fa14 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:34.155353 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 528/1057] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:57:34.161192 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941d643-07b1-4967-ac58-5deb39492a5d None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:34.161665 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941d643-07b1-4967-ac58-5deb39492a5d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:34.161771 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941d643-07b1-4967-ac58-5deb39492a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:34.174499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:34.230859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-e941d643-07b1-4967-ac58-5deb39492a5d None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:57:34.232051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e941d643-07b1-4967-ac58-5deb39492a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:34.341371 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 530/1058] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:57:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:57:34.638825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f921a-f8d4-4b83-8d9a-a960c4f4fa14 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:57:34.640653 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-874f921a-f8d4-4b83-8d9a-a960c4f4fa14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:34.804950 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:34.819237 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f921a-f8d4-4b83-8d9a-a960c4f4fa14 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:57:34.947712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea977e4-d0bd-4c3a-824c-f3492a96ab23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:57:34.948332 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea977e4-d0bd-4c3a-824c-f3492a96ab23 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:34.948609 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea977e4-d0bd-4c3a-824c-f3492a96ab23 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:34.948905 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea977e4-d0bd-4c3a-824c-f3492a96ab23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:34.950515 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9ea977e4-d0bd-4c3a-824c-f3492a96ab23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:35.048043 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea977e4-d0bd-4c3a-824c-f3492a96ab23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:57:35.050206 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea977e4-d0bd-4c3a-824c-f3492a96ab23 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:57:35.062749 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:35.071806 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 529/1059] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:57:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:57:37.256420 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011be7bc-5b6c-469b-bad8-360167c5ac00 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:37.257004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011be7bc-5b6c-469b-bad8-360167c5ac00 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:37.257160 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011be7bc-5b6c-469b-bad8-360167c5ac00 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:37.258929 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 531/1060] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:57:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:57:37.264457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae9247-6e31-4407-b9bf-3707926b1ad8 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:37.264868 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae9247-6e31-4407-b9bf-3707926b1ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:37.265079 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae9247-6e31-4407-b9bf-3707926b1ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:37.338598 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-62ae9247-6e31-4407-b9bf-3707926b1ad8 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:57:37.340112 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-62ae9247-6e31-4407-b9bf-3707926b1ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:37.457201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 530/1061] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:57:37.750821 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011be7bc-5b6c-469b-bad8-360167c5ac00 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:57:37.753693 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-011be7bc-5b6c-469b-bad8-360167c5ac00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:37.860572 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:37.939689 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011be7bc-5b6c-469b-bad8-360167c5ac00 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:57:38.063160 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be9613-0461-4a97-b5e6-eafd52f355e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:57:38.063685 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be9613-0461-4a97-b5e6-eafd52f355e2 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:38.063968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be9613-0461-4a97-b5e6-eafd52f355e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:38.064804 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be9613-0461-4a97-b5e6-eafd52f355e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:38.066467 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f6be9613-0461-4a97-b5e6-eafd52f355e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:38.118052 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:38.165018 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be9613-0461-4a97-b5e6-eafd52f355e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:57:38.166693 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be9613-0461-4a97-b5e6-eafd52f355e2 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:57:38.185500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 532/1062] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:57:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:57:40.530224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f25345-3d1d-45fe-a3d6-6111552e577a None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:40.530897 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f25345-3d1d-45fe-a3d6-6111552e577a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:40.530897 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f25345-3d1d-45fe-a3d6-6111552e577a None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:40.532103 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 531/1063] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:57:40.537894 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fa979-d45a-4ee5-80c9-09f054eba810 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:40.538309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fa979-d45a-4ee5-80c9-09f054eba810 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:40.538468 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fa979-d45a-4ee5-80c9-09f054eba810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:40.606631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-6c8fa979-d45a-4ee5-80c9-09f054eba810 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:57:40.607828 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6c8fa979-d45a-4ee5-80c9-09f054eba810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:40.706920 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 533/1064] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:57:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:57:41.003854 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f25345-3d1d-45fe-a3d6-6111552e577a None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:57:41.005680 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e7f25345-3d1d-45fe-a3d6-6111552e577a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:41.080876 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:41.149005 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f25345-3d1d-45fe-a3d6-6111552e577a None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:57:41.263356 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334cdd46-f64a-40de-b091-635ba56c36ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:57:41.263877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cdd46-f64a-40de-b091-635ba56c36ae service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:41.264153 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cdd46-f64a-40de-b091-635ba56c36ae service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:41.264421 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cdd46-f64a-40de-b091-635ba56c36ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:41.266039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-334cdd46-f64a-40de-b091-635ba56c36ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:41.309974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:41.362095 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334cdd46-f64a-40de-b091-635ba56c36ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:57:41.363694 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334cdd46-f64a-40de-b091-635ba56c36ae service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:57:41.382215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 532/1065] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:57:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:57:42.246307 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0e694-a678-4a67-8066-239e9558698a None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:42.246864 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0e694-a678-4a67-8066-239e9558698a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:42.246864 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0e694-a678-4a67-8066-239e9558698a None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:42.248223 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 534/1066] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:57:42.253656 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c61389-6901-46a3-922c-87f36cd451ac None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:42.254110 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c61389-6901-46a3-922c-87f36cd451ac None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:42.254571 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c61389-6901-46a3-922c-87f36cd451ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:42.324415 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-76c61389-6901-46a3-922c-87f36cd451ac None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:57:42.325725 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-76c61389-6901-46a3-922c-87f36cd451ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:42.422103 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 533/1067] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:57:42.723269 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676d2d6-c82c-4828-ae55-d773e7225545 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:42.723577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676d2d6-c82c-4828-ae55-d773e7225545 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:42.723848 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676d2d6-c82c-4828-ae55-d773e7225545 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:42.725197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 535/1068] 10.222.0.20 () {54 vars in 962 bytes} [Thu Oct 20 05:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:57:42.730442 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe1bfd-ee7b-4865-b284-2a2c46157dc0 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:42.730729 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe1bfd-ee7b-4865-b284-2a2c46157dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:42.730995 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe1bfd-ee7b-4865-b284-2a2c46157dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:42.747288 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:42.806523 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-01fe1bfd-ee7b-4865-b284-2a2c46157dc0 None None] MFA Rules not processed for user `3dcfe5244bd04c00b6dde1d6f795d4ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:57:42.807702 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-01fe1bfd-ee7b-4865-b284-2a2c46157dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:42.882303 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 534/1069] 10.222.0.20 () {58 vars in 1059 bytes} [Thu Oct 20 05:57:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:57:42.891377 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676d2d6-c82c-4828-ae55-d773e7225545 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:57:42.893625 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1676d2d6-c82c-4828-ae55-d773e7225545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:42.925356 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:43.000902 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676d2d6-c82c-4828-ae55-d773e7225545 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:57:43.077326 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca51f4b-e7bf-40f0-8487-89f4ae9c7cad service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dcfe5244bd04c00b6dde1d6f795d4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:57:43.077938 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca51f4b-e7bf-40f0-8487-89f4ae9c7cad service glance] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:43.078215 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca51f4b-e7bf-40f0-8487-89f4ae9c7cad service glance] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:43.078484 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca51f4b-e7bf-40f0-8487-89f4ae9c7cad service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:43.080193 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5ca51f4b-e7bf-40f0-8487-89f4ae9c7cad service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:43.158250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:43.190742 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca51f4b-e7bf-40f0-8487-89f4ae9c7cad service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:57:43.192181 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca51f4b-e7bf-40f0-8487-89f4ae9c7cad service glance] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:57:43.211339 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 536/1070] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:57:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:57:43.576828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267981c-52e0-420f-a509-adba365ef78e None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:43.577306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267981c-52e0-420f-a509-adba365ef78e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:43.577419 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267981c-52e0-420f-a509-adba365ef78e None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:43.578793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 535/1071] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:57:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:57:43.585239 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6108ecd-883e-4a72-ad3f-c8c5b8458e8b None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:43.585749 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6108ecd-883e-4a72-ad3f-c8c5b8458e8b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:43.585862 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6108ecd-883e-4a72-ad3f-c8c5b8458e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:43.663040 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-a6108ecd-883e-4a72-ad3f-c8c5b8458e8b None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:57:43.664356 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a6108ecd-883e-4a72-ad3f-c8c5b8458e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:43.764039 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 537/1072] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:57:43.958376 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643208aa-d43e-4bfd-8b63-a85d88ee178b None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:43.958662 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643208aa-d43e-4bfd-8b63-a85d88ee178b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:43.958989 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643208aa-d43e-4bfd-8b63-a85d88ee178b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:44.026511 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-643208aa-d43e-4bfd-8b63-a85d88ee178b None None] MFA Rules not processed for user `7a59f547a20e48ee88e3e620ba1e87f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:57:44.027702 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-643208aa-d43e-4bfd-8b63-a85d88ee178b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:44.058447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6108ecd-883e-4a72-ad3f-c8c5b8458e8b None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:57:44.059799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a6108ecd-883e-4a72-ad3f-c8c5b8458e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:44.085272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:44.100784 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:44.118167 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 536/1073] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Oct 20 05:57:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:57:44.162564 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c643d-8dec-418e-a3b4-cdebcd206568 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:44.162845 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c643d-8dec-418e-a3b4-cdebcd206568 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:44.163113 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c643d-8dec-418e-a3b4-cdebcd206568 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:44.164474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 537/1074] 10.222.0.20 () {54 vars in 943 bytes} [Thu Oct 20 05:57:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:57:44.170744 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f9f1e-ac21-4770-9a05-aecba0e0f087 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:44.171094 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f9f1e-ac21-4770-9a05-aecba0e0f087 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:44.171433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f9f1e-ac21-4770-9a05-aecba0e0f087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:44.180627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6108ecd-883e-4a72-ad3f-c8c5b8458e8b None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:57:44.249945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d93f9f1e-ac21-4770-9a05-aecba0e0f087 None None] MFA Rules not processed for user `eeeab506ab5c4d6386fc899086f0abf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:57:44.251185 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d93f9f1e-ac21-4770-9a05-aecba0e0f087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:44.304334 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71dd67f-279a-4478-8363-5aa50294ded5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:57:44.305133 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71dd67f-279a-4478-8363-5aa50294ded5 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:44.305257 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71dd67f-279a-4478-8363-5aa50294ded5 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:44.305535 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71dd67f-279a-4478-8363-5aa50294ded5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:44.307369 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f71dd67f-279a-4478-8363-5aa50294ded5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:44.343017 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:44.358081 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 538/1075] 10.222.0.20 () {58 vars in 1040 bytes} [Thu Oct 20 05:57:44 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 05:57:44.368086 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f9f1e-ac21-4770-9a05-aecba0e0f087 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:57:44.377666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f9f1e-ac21-4770-9a05-aecba0e0f087 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:57:44.400086 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71dd67f-279a-4478-8363-5aa50294ded5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:57:44.401638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71dd67f-279a-4478-8363-5aa50294ded5 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:57:44.404148 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:44.419834 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 538/1076] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:57:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 05:57:44.492268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fdb42e-d220-444a-8eec-81dc18500eb9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeeab506ab5c4d6386fc899086f0abf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:57:44.492918 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fdb42e-d220-444a-8eec-81dc18500eb9 service swift] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:44.493195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fdb42e-d220-444a-8eec-81dc18500eb9 service swift] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:44.493461 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fdb42e-d220-444a-8eec-81dc18500eb9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:44.495293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-92fdb42e-d220-444a-8eec-81dc18500eb9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:44.588493 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fdb42e-d220-444a-8eec-81dc18500eb9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:57:44.590135 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fdb42e-d220-444a-8eec-81dc18500eb9 service swift] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:57:44.598951 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 539/1077] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Oct 20 05:57:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 20 05:57:46.651010 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245e1c8-7761-4519-b7e4-85bec21f17bd None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:46.651563 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245e1c8-7761-4519-b7e4-85bec21f17bd None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:46.651563 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245e1c8-7761-4519-b7e4-85bec21f17bd None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:46.656867 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 539/1078] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:57:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:57:46.659379 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee644c1-df6b-49b8-b864-37e397c3fb09 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:46.659379 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee644c1-df6b-49b8-b864-37e397c3fb09 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:46.659379 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee644c1-df6b-49b8-b864-37e397c3fb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:46.741569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-4ee644c1-df6b-49b8-b864-37e397c3fb09 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:57:46.742880 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4ee644c1-df6b-49b8-b864-37e397c3fb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:46.750482 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:46.851103 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 540/1079] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:57:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:57:47.153004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245e1c8-7761-4519-b7e4-85bec21f17bd None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:57:47.154592 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d245e1c8-7761-4519-b7e4-85bec21f17bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:47.347505 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245e1c8-7761-4519-b7e4-85bec21f17bd None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:57:47.362789 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:47.495973 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408f5b6-df5f-4ec3-9ed6-fb2f7cedaaa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:57:47.497120 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408f5b6-df5f-4ec3-9ed6-fb2f7cedaaa2 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:47.497404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408f5b6-df5f-4ec3-9ed6-fb2f7cedaaa2 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:47.497685 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408f5b6-df5f-4ec3-9ed6-fb2f7cedaaa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:47.499654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7408f5b6-df5f-4ec3-9ed6-fb2f7cedaaa2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:47.621928 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408f5b6-df5f-4ec3-9ed6-fb2f7cedaaa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:57:47.623694 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408f5b6-df5f-4ec3-9ed6-fb2f7cedaaa2 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:57:47.647996 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 540/1080] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:57:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:57:49.888816 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8aeb92-9ba5-4177-88bc-ffd5fe02bd88 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:49.889393 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8aeb92-9ba5-4177-88bc-ffd5fe02bd88 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:49.889393 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8aeb92-9ba5-4177-88bc-ffd5fe02bd88 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:49.893040 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 541/1081] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:57:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:57:49.895964 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00c62a-92ac-4f1f-96a5-c6263207125d None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:49.896375 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00c62a-92ac-4f1f-96a5-c6263207125d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:49.896555 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00c62a-92ac-4f1f-96a5-c6263207125d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:49.917130 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:49.971533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-bd00c62a-92ac-4f1f-96a5-c6263207125d None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:57:49.972766 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-bd00c62a-92ac-4f1f-96a5-c6263207125d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:50.079929 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 541/1082] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:57:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:57:50.370299 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8aeb92-9ba5-4177-88bc-ffd5fe02bd88 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:57:50.371841 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3b8aeb92-9ba5-4177-88bc-ffd5fe02bd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:50.539837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:50.552015 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8aeb92-9ba5-4177-88bc-ffd5fe02bd88 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:57:50.684121 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e98840-69c8-4fef-99b9-ca8fee885f1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:57:50.684942 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e98840-69c8-4fef-99b9-ca8fee885f1c service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:50.685290 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e98840-69c8-4fef-99b9-ca8fee885f1c service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:50.685661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e98840-69c8-4fef-99b9-ca8fee885f1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:50.687452 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-67e98840-69c8-4fef-99b9-ca8fee885f1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:50.793595 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e98840-69c8-4fef-99b9-ca8fee885f1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:57:50.795180 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e98840-69c8-4fef-99b9-ca8fee885f1c service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:57:50.805969 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:50.817561 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 542/1083] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:57:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:57:53.083837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6cf46-ab87-4965-ba2d-8212bd8a5884 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:53.084405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6cf46-ab87-4965-ba2d-8212bd8a5884 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:53.084405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6cf46-ab87-4965-ba2d-8212bd8a5884 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:53.088588 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 542/1084] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:57:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:57:53.094185 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55ac3c-8130-4dba-b7e2-ef5da4dbd210 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:53.095291 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55ac3c-8130-4dba-b7e2-ef5da4dbd210 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:53.095385 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55ac3c-8130-4dba-b7e2-ef5da4dbd210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:53.170141 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-0c55ac3c-8130-4dba-b7e2-ef5da4dbd210 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:57:53.171693 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-0c55ac3c-8130-4dba-b7e2-ef5da4dbd210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:53.277853 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 543/1085] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:57:53.591742 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6cf46-ab87-4965-ba2d-8212bd8a5884 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:57:53.593297 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-72f6cf46-ab87-4965-ba2d-8212bd8a5884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:53.699364 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:53.759613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6cf46-ab87-4965-ba2d-8212bd8a5884 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:57:53.883637 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5945fe-b6c8-47e3-880c-f9abe8397aab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:57:53.884422 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5945fe-b6c8-47e3-880c-f9abe8397aab service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:57:53.884779 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5945fe-b6c8-47e3-880c-f9abe8397aab service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:57:53.884995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5945fe-b6c8-47e3-880c-f9abe8397aab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:57:53.886667 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fb5945fe-b6c8-47e3-880c-f9abe8397aab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:57:53.953558 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:57:53.989062 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5945fe-b6c8-47e3-880c-f9abe8397aab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:57:53.990774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5945fe-b6c8-47e3-880c-f9abe8397aab service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:57:54.012438 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 543/1086] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:57:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:58:03.276192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=67 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:04.010109 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=20 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:07.615411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d1ff8-a8f0-42d6-8d98-0ec79dd678b0 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:07.615925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d1ff8-a8f0-42d6-8d98-0ec79dd678b0 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:07.615925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d1ff8-a8f0-42d6-8d98-0ec79dd678b0 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:07.617281 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 544/1087] 10.222.0.20 () {54 vars in 918 bytes} [Thu Oct 20 05:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:58:07.622387 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f128cd-a9e0-45dd-98d4-375ca5144767 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:07.622829 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f128cd-a9e0-45dd-98d4-375ca5144767 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:07.622937 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f128cd-a9e0-45dd-98d4-375ca5144767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:07.698979 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-68f128cd-a9e0-45dd-98d4-375ca5144767 None None] MFA Rules not processed for user `cfbaaf32475a4c82a34948602d8f4e35`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:58:07.700168 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-68f128cd-a9e0-45dd-98d4-375ca5144767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:07.804026 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 544/1088] 10.222.0.20 () {58 vars in 1015 bytes} [Thu Oct 20 05:58:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:58:07.818561 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f18e0-5709-4ade-996e-0fba5f3bd37a None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:07.818841 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f18e0-5709-4ade-996e-0fba5f3bd37a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:07.819108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f18e0-5709-4ade-996e-0fba5f3bd37a None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:07.820468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 545/1089] 10.222.0.20 () {54 vars in 918 bytes} [Thu Oct 20 05:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:58:07.824944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed34b01-b5c7-481e-9bc4-fa3fc25619c4 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:07.825232 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed34b01-b5c7-481e-9bc4-fa3fc25619c4 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:07.825517 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed34b01-b5c7-481e-9bc4-fa3fc25619c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:07.898023 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:07.901144 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-5ed34b01-b5c7-481e-9bc4-fa3fc25619c4 None None] MFA Rules not processed for user `ae2c1c886cd94d9f90082f66e7e8c9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:58:07.902926 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5ed34b01-b5c7-481e-9bc4-fa3fc25619c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:08.017599 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 545/1090] 10.222.0.20 () {58 vars in 1015 bytes} [Thu Oct 20 05:58:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:58:10.602003 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7946d8-836a-4feb-9016-2bd0771c1133 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:10.602540 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7946d8-836a-4feb-9016-2bd0771c1133 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:10.602540 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7946d8-836a-4feb-9016-2bd0771c1133 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:10.603798 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 546/1091] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 05:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:58:10.609017 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8949a-3c35-4ca0-bc60-1a9edd589857 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:10.609575 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8949a-3c35-4ca0-bc60-1a9edd589857 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:10.609575 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8949a-3c35-4ca0-bc60-1a9edd589857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:10.674974 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-ecf8949a-3c35-4ca0-bc60-1a9edd589857 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:58:10.676180 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ecf8949a-3c35-4ca0-bc60-1a9edd589857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:10.731565 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:10.773895 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 546/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 05:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:58:11.062028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7946d8-836a-4feb-9016-2bd0771c1133 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:58:11.063994 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4d7946d8-836a-4feb-9016-2bd0771c1133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:11.208674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7946d8-836a-4feb-9016-2bd0771c1133 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:58:11.280425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:11.320536 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c8b65-44a7-4fa9-ae5f-133be187454c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:58:11.321191 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c8b65-44a7-4fa9-ae5f-133be187454c service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:11.321458 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c8b65-44a7-4fa9-ae5f-133be187454c service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:11.321719 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c8b65-44a7-4fa9-ae5f-133be187454c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:11.323288 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e99c8b65-44a7-4fa9-ae5f-133be187454c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:11.410286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99c8b65-44a7-4fa9-ae5f-133be187454c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:58:11.411862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99c8b65-44a7-4fa9-ae5f-133be187454c service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:58:11.429264 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 547/1093] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:58:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:58:12.564423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a6dd9-83e2-4509-b0bd-d2dd6076de6d None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:12.564999 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a6dd9-83e2-4509-b0bd-d2dd6076de6d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:12.564999 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a6dd9-83e2-4509-b0bd-d2dd6076de6d None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:12.566277 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 547/1094] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 05:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:58:12.572321 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b940a4b-dc2a-4486-8767-c54641e02c02 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:12.572781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b940a4b-dc2a-4486-8767-c54641e02c02 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:12.572892 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b940a4b-dc2a-4486-8767-c54641e02c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:12.648528 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-6b940a4b-dc2a-4486-8767-c54641e02c02 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:58:12.649791 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6b940a4b-dc2a-4486-8767-c54641e02c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:12.677696 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:12.747224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 548/1095] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 05:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:58:12.903536 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a6dd9-83e2-4509-b0bd-d2dd6076de6d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:58:12.905078 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1d2a6dd9-83e2-4509-b0bd-d2dd6076de6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:13.050001 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a6dd9-83e2-4509-b0bd-d2dd6076de6d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:58:13.089967 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:13.164104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be50a42-8003-4d45-944b-26243d354603 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:58:13.164729 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be50a42-8003-4d45-944b-26243d354603 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:13.165015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be50a42-8003-4d45-944b-26243d354603 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:13.165285 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be50a42-8003-4d45-944b-26243d354603 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:13.166868 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3be50a42-8003-4d45-944b-26243d354603 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:13.254633 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be50a42-8003-4d45-944b-26243d354603 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:58:13.256209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be50a42-8003-4d45-944b-26243d354603 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:58:13.273709 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 548/1096] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:58:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:58:14.408074 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac295ffb-4084-4341-a32a-63ecd1f04d3b None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:14.408614 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac295ffb-4084-4341-a32a-63ecd1f04d3b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:14.408614 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac295ffb-4084-4341-a32a-63ecd1f04d3b None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:14.409906 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 549/1097] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 05:58:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:58:14.416030 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e29ef7-a55b-43d6-aa47-be417d0c7c9c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:14.416441 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e29ef7-a55b-43d6-aa47-be417d0c7c9c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:14.416566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e29ef7-a55b-43d6-aa47-be417d0c7c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:14.470795 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:14.493826 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c4e29ef7-a55b-43d6-aa47-be417d0c7c9c None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:58:14.495031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c4e29ef7-a55b-43d6-aa47-be417d0c7c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:14.589417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 549/1098] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 05:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:58:14.742598 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac295ffb-4084-4341-a32a-63ecd1f04d3b None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:58:14.744052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ac295ffb-4084-4341-a32a-63ecd1f04d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:14.884662 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac295ffb-4084-4341-a32a-63ecd1f04d3b None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:58:14.896396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:14.995286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf528a-0396-4b3e-af71-33a1ab5fa92d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:58:14.995812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf528a-0396-4b3e-af71-33a1ab5fa92d service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:14.996084 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf528a-0396-4b3e-af71-33a1ab5fa92d service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:14.996350 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf528a-0396-4b3e-af71-33a1ab5fa92d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:14.997959 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e8cf528a-0396-4b3e-af71-33a1ab5fa92d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:15.084535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf528a-0396-4b3e-af71-33a1ab5fa92d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:58:15.086148 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf528a-0396-4b3e-af71-33a1ab5fa92d service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:58:15.104474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 550/1099] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:58:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:58:16.237592 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d93f15-e87d-44ad-9d02-698bc554ce18 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:16.238159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d93f15-e87d-44ad-9d02-698bc554ce18 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:16.238159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d93f15-e87d-44ad-9d02-698bc554ce18 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:16.239514 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 550/1100] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 05:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:58:16.245713 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275aec8-dc30-4e19-9497-0d1026beec31 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:16.246105 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275aec8-dc30-4e19-9497-0d1026beec31 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:16.246253 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275aec8-dc30-4e19-9497-0d1026beec31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:16.266969 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:16.322425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-4275aec8-dc30-4e19-9497-0d1026beec31 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:58:16.323622 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4275aec8-dc30-4e19-9497-0d1026beec31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:16.419332 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 551/1101] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 05:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:58:16.568651 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d93f15-e87d-44ad-9d02-698bc554ce18 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:58:16.570212 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-84d93f15-e87d-44ad-9d02-698bc554ce18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:16.679890 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:16.715545 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d93f15-e87d-44ad-9d02-698bc554ce18 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:58:16.824878 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5304ad4-8cb2-4658-b2f4-e573e2227dbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:58:16.825476 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5304ad4-8cb2-4658-b2f4-e573e2227dbc service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:16.825747 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5304ad4-8cb2-4658-b2f4-e573e2227dbc service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:16.826009 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5304ad4-8cb2-4658-b2f4-e573e2227dbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:16.828109 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d5304ad4-8cb2-4658-b2f4-e573e2227dbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:16.903570 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:16.917343 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5304ad4-8cb2-4658-b2f4-e573e2227dbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:58:16.918924 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5304ad4-8cb2-4658-b2f4-e573e2227dbc service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:58:16.936724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 551/1102] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:58:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:58:18.057208 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0de888-6a3e-4706-ac24-9eca45d59c5c None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:18.057737 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0de888-6a3e-4706-ac24-9eca45d59c5c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:18.057737 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0de888-6a3e-4706-ac24-9eca45d59c5c None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:18.058988 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 552/1103] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 05:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:58:18.065893 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe4374-5129-40a0-b385-ed8df57fb176 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:18.066353 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe4374-5129-40a0-b385-ed8df57fb176 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:18.066450 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe4374-5129-40a0-b385-ed8df57fb176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:18.143470 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-fabe4374-5129-40a0-b385-ed8df57fb176 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:58:18.144684 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fabe4374-5129-40a0-b385-ed8df57fb176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:18.244115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 552/1104] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 05:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:58:18.396466 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0de888-6a3e-4706-ac24-9eca45d59c5c None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:58:18.398074 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-da0de888-6a3e-4706-ac24-9eca45d59c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:18.480349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:18.541628 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0de888-6a3e-4706-ac24-9eca45d59c5c None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:58:18.651915 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6cff9-3b7f-4119-92dd-9d450c12de0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:58:18.652436 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6cff9-3b7f-4119-92dd-9d450c12de0d service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:18.652740 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6cff9-3b7f-4119-92dd-9d450c12de0d service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:18.653016 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6cff9-3b7f-4119-92dd-9d450c12de0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:18.654612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-add6cff9-3b7f-4119-92dd-9d450c12de0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:18.702786 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:18.744245 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add6cff9-3b7f-4119-92dd-9d450c12de0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:58:18.745857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add6cff9-3b7f-4119-92dd-9d450c12de0d service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:58:18.763697 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 553/1105] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:58:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:58:19.821589 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83e621-7b86-4d9d-9774-c1a8f8ef671d None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:19.822164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83e621-7b86-4d9d-9774-c1a8f8ef671d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:19.822164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83e621-7b86-4d9d-9774-c1a8f8ef671d None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:19.823458 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 553/1106] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 05:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:58:19.830008 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4176612-28a7-401c-87ae-b90281a2a951 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:19.830443 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4176612-28a7-401c-87ae-b90281a2a951 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:19.830555 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4176612-28a7-401c-87ae-b90281a2a951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:19.906795 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-f4176612-28a7-401c-87ae-b90281a2a951 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:58:19.908028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f4176612-28a7-401c-87ae-b90281a2a951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:20.001694 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 554/1107] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 05:58:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:58:20.154606 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83e621-7b86-4d9d-9774-c1a8f8ef671d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:58:20.156599 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ec83e621-7b86-4d9d-9774-c1a8f8ef671d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:20.205098 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:20.300861 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83e621-7b86-4d9d-9774-c1a8f8ef671d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:58:20.412226 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc3245f-4aa0-4151-9f98-a648227733b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:58:20.412951 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3245f-4aa0-4151-9f98-a648227733b9 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:20.413234 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3245f-4aa0-4151-9f98-a648227733b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:20.413506 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3245f-4aa0-4151-9f98-a648227733b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:20.415128 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3fc3245f-4aa0-4151-9f98-a648227733b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:20.422828 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:20.508883 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc3245f-4aa0-4151-9f98-a648227733b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:58:20.510464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc3245f-4aa0-4151-9f98-a648227733b9 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:58:20.528508 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 554/1108] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:58:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:58:21.592449 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea120fba-cc3f-4a4b-8b03-8756081f1fdd None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:21.592979 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea120fba-cc3f-4a4b-8b03-8756081f1fdd None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:21.592979 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea120fba-cc3f-4a4b-8b03-8756081f1fdd None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:21.594555 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 555/1109] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 05:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:58:21.600860 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5e6b0-4e6e-49cf-8331-bf71b758aab7 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:21.601305 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5e6b0-4e6e-49cf-8331-bf71b758aab7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:21.601404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5e6b0-4e6e-49cf-8331-bf71b758aab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:21.676530 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-6fb5e6b0-4e6e-49cf-8331-bf71b758aab7 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:58:21.677737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6fb5e6b0-4e6e-49cf-8331-bf71b758aab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:21.744415 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:21.773983 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 555/1110] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 05:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 05:58:21.923608 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea120fba-cc3f-4a4b-8b03-8756081f1fdd None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:58:21.925161 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ea120fba-cc3f-4a4b-8b03-8756081f1fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:21.942790 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:22.069886 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea120fba-cc3f-4a4b-8b03-8756081f1fdd None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:58:22.154886 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:22.181011 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cca6e-7fa4-43e5-91c8-8f67b57f6b14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:58:22.181543 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cca6e-7fa4-43e5-91c8-8f67b57f6b14 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:22.181818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cca6e-7fa4-43e5-91c8-8f67b57f6b14 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:22.182091 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cca6e-7fa4-43e5-91c8-8f67b57f6b14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:22.183672 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-826cca6e-7fa4-43e5-91c8-8f67b57f6b14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:22.271078 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cca6e-7fa4-43e5-91c8-8f67b57f6b14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:58:22.272665 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cca6e-7fa4-43e5-91c8-8f67b57f6b14 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:58:22.290660 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 556/1111] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:58:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:58:23.348381 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33320d5e-427b-4583-96c7-b126a00516a3 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:23.348883 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33320d5e-427b-4583-96c7-b126a00516a3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:23.348992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33320d5e-427b-4583-96c7-b126a00516a3 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:23.350283 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 556/1112] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 05:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:58:23.356401 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff00f3-0444-428b-b669-a8476145aa33 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:23.356816 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff00f3-0444-428b-b669-a8476145aa33 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:23.356985 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff00f3-0444-428b-b669-a8476145aa33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:23.433277 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-f5ff00f3-0444-428b-b669-a8476145aa33 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:58:23.434465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f5ff00f3-0444-428b-b669-a8476145aa33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:23.474985 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:23.529354 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 557/1113] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 05:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:58:23.684644 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33320d5e-427b-4583-96c7-b126a00516a3 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:58:23.686228 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-33320d5e-427b-4583-96c7-b126a00516a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:23.829358 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33320d5e-427b-4583-96c7-b126a00516a3 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:58:23.882234 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:23.940446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43aa1b-1c96-412c-876e-5d50149d03f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:58:23.941070 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43aa1b-1c96-412c-876e-5d50149d03f5 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:23.941342 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43aa1b-1c96-412c-876e-5d50149d03f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:23.941601 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43aa1b-1c96-412c-876e-5d50149d03f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:23.943168 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-bf43aa1b-1c96-412c-876e-5d50149d03f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:24.032985 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43aa1b-1c96-412c-876e-5d50149d03f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:58:24.034605 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43aa1b-1c96-412c-876e-5d50149d03f5 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:58:24.052545 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 557/1114] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:58:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:58:25.107504 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41ae46-e2d1-4657-ae43-21e8d28b2bac None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:25.108049 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41ae46-e2d1-4657-ae43-21e8d28b2bac None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:25.108049 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41ae46-e2d1-4657-ae43-21e8d28b2bac None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:25.110089 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 558/1115] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 05:58:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:58:25.116361 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8ce8b-fcfc-421e-a640-6840058c888d None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:25.116856 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8ce8b-fcfc-421e-a640-6840058c888d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:25.116956 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8ce8b-fcfc-421e-a640-6840058c888d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:25.192128 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-18b8ce8b-fcfc-421e-a640-6840058c888d None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:58:25.193343 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-18b8ce8b-fcfc-421e-a640-6840058c888d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:25.195312 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:25.289206 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 558/1116] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 05:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:58:25.437189 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41ae46-e2d1-4657-ae43-21e8d28b2bac None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:58:25.438711 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ab41ae46-e2d1-4657-ae43-21e8d28b2bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:25.585549 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41ae46-e2d1-4657-ae43-21e8d28b2bac None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:58:25.612794 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:25.698187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d116d32-5083-42d7-894e-96966bfbffd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:58:25.698718 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d116d32-5083-42d7-894e-96966bfbffd4 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:25.698995 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d116d32-5083-42d7-894e-96966bfbffd4 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:25.699266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d116d32-5083-42d7-894e-96966bfbffd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:25.700889 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3d116d32-5083-42d7-894e-96966bfbffd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:25.787847 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d116d32-5083-42d7-894e-96966bfbffd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:58:25.789469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d116d32-5083-42d7-894e-96966bfbffd4 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:58:25.807476 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 559/1117] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:58:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:58:27.870250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a06c5-e88c-4a1f-af25-30ff3744759d None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:27.870799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a06c5-e88c-4a1f-af25-30ff3744759d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:27.870799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a06c5-e88c-4a1f-af25-30ff3744759d None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:27.872114 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 559/1118] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 05:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 05:58:27.878352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0bd92-8c3d-42a7-aab4-49849b7cb1e7 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:27.878777 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0bd92-8c3d-42a7-aab4-49849b7cb1e7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:27.878961 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0bd92-8c3d-42a7-aab4-49849b7cb1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:27.920050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:27.946365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-61b0bd92-8c3d-42a7-aab4-49849b7cb1e7 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:58:27.947599 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-61b0bd92-8c3d-42a7-aab4-49849b7cb1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:28.045186 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 560/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 05:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:58:28.341648 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a06c5-e88c-4a1f-af25-30ff3744759d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:58:28.344082 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ff9a06c5-e88c-4a1f-af25-30ff3744759d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:28.447435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a06c5-e88c-4a1f-af25-30ff3744759d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:58:28.480424 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:28.522907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db8b81-1951-40b1-b058-e5ef2aa5ebff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:58:28.523513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db8b81-1951-40b1-b058-e5ef2aa5ebff service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:28.523793 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db8b81-1951-40b1-b058-e5ef2aa5ebff service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:28.524064 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db8b81-1951-40b1-b058-e5ef2aa5ebff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:28.525684 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b1db8b81-1951-40b1-b058-e5ef2aa5ebff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:28.634089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1db8b81-1951-40b1-b058-e5ef2aa5ebff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:58:28.635522 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1db8b81-1951-40b1-b058-e5ef2aa5ebff service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:58:28.653867 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 560/1120] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:58:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:58:30.922770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c60ea6-d7fe-427d-a21f-b196bbcccfcf None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:30.923368 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c60ea6-d7fe-427d-a21f-b196bbcccfcf None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:30.923368 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c60ea6-d7fe-427d-a21f-b196bbcccfcf None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:30.924674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 561/1121] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 05:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:58:30.929798 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783615ae-d10d-4669-9070-f8db5994966b None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:30.930239 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783615ae-d10d-4669-9070-f8db5994966b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:30.930374 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783615ae-d10d-4669-9070-f8db5994966b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:30.996086 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-783615ae-d10d-4669-9070-f8db5994966b None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:58:30.997288 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-783615ae-d10d-4669-9070-f8db5994966b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:31.009981 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:31.093304 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 561/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 05:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:58:31.382655 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c60ea6-d7fe-427d-a21f-b196bbcccfcf None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:58:31.384230 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-39c60ea6-d7fe-427d-a21f-b196bbcccfcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:31.486292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c60ea6-d7fe-427d-a21f-b196bbcccfcf None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:58:31.498935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:31.559625 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528fad14-fe5d-46c2-950f-efa8b197bc2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:58:31.560150 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fad14-fe5d-46c2-950f-efa8b197bc2a service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:31.560423 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fad14-fe5d-46c2-950f-efa8b197bc2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:31.560714 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fad14-fe5d-46c2-950f-efa8b197bc2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:31.562405 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-528fad14-fe5d-46c2-950f-efa8b197bc2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:31.671212 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528fad14-fe5d-46c2-950f-efa8b197bc2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:58:31.672644 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528fad14-fe5d-46c2-950f-efa8b197bc2a service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:58:31.690006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 562/1123] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:58:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:58:34.148030 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888295fb-7c11-466d-ab0e-c43ebef14825 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:34.148597 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888295fb-7c11-466d-ab0e-c43ebef14825 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:34.148597 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888295fb-7c11-466d-ab0e-c43ebef14825 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:34.149973 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 562/1124] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 05:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:58:34.155732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a014d-c9b6-44b8-aff8-2b01c57f7dbd None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:34.155732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a014d-c9b6-44b8-aff8-2b01c57f7dbd None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:34.156181 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a014d-c9b6-44b8-aff8-2b01c57f7dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:34.205590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:34.223420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-181a014d-c9b6-44b8-aff8-2b01c57f7dbd None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:58:34.224601 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-181a014d-c9b6-44b8-aff8-2b01c57f7dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:34.319268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 563/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 05:58:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 05:58:34.610927 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888295fb-7c11-466d-ab0e-c43ebef14825 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:58:34.612450 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-888295fb-7c11-466d-ab0e-c43ebef14825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:34.714119 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888295fb-7c11-466d-ab0e-c43ebef14825 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:58:34.754082 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:34.786732 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8134e8-ff59-4d65-bfe3-2c689632812f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:58:34.787327 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8134e8-ff59-4d65-bfe3-2c689632812f service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:34.787609 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8134e8-ff59-4d65-bfe3-2c689632812f service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:34.787879 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8134e8-ff59-4d65-bfe3-2c689632812f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:34.789491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-bc8134e8-ff59-4d65-bfe3-2c689632812f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:34.899683 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8134e8-ff59-4d65-bfe3-2c689632812f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:58:34.901144 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8134e8-ff59-4d65-bfe3-2c689632812f service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:58:34.918948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 563/1126] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:58:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:58:37.202457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f723ad-f595-4136-b843-244608448771 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:37.202996 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f723ad-f595-4136-b843-244608448771 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:37.202996 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f723ad-f595-4136-b843-244608448771 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:37.204369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 564/1127] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 05:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:58:37.209801 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7bbf0-903c-409f-9bab-7a206b93209b None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:37.210202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7bbf0-903c-409f-9bab-7a206b93209b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:37.210359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7bbf0-903c-409f-9bab-7a206b93209b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:37.278045 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-22f7bbf0-903c-409f-9bab-7a206b93209b None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:58:37.279788 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-22f7bbf0-903c-409f-9bab-7a206b93209b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:37.302629 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:37.377178 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 564/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 05:58:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:58:37.675529 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f723ad-f595-4136-b843-244608448771 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:58:37.677102 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-24f723ad-f595-4136-b843-244608448771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:37.778152 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f723ad-f595-4136-b843-244608448771 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:58:37.798752 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:37.850266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e370869-e6f3-4a2d-894f-43622a6a6b60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:58:37.850861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e370869-e6f3-4a2d-894f-43622a6a6b60 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:37.851134 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e370869-e6f3-4a2d-894f-43622a6a6b60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:37.851398 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e370869-e6f3-4a2d-894f-43622a6a6b60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:37.853000 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5e370869-e6f3-4a2d-894f-43622a6a6b60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:37.958028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e370869-e6f3-4a2d-894f-43622a6a6b60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:58:37.959372 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e370869-e6f3-4a2d-894f-43622a6a6b60 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:58:37.977624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 565/1129] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:58:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:58:40.176718 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f501ce-5c9d-4697-b1ff-2de3bb188b17 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:40.177186 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f501ce-5c9d-4697-b1ff-2de3bb188b17 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:40.177291 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f501ce-5c9d-4697-b1ff-2de3bb188b17 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:40.178573 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 565/1130] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 05:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:58:40.183962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28483fe-292a-44ff-b018-a98c75f0a4e9 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:40.184398 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28483fe-292a-44ff-b018-a98c75f0a4e9 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:40.184495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28483fe-292a-44ff-b018-a98c75f0a4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:40.247658 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:40.250254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d28483fe-292a-44ff-b018-a98c75f0a4e9 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:58:40.252002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d28483fe-292a-44ff-b018-a98c75f0a4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:40.348242 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 566/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 05:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:58:40.635371 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f501ce-5c9d-4697-b1ff-2de3bb188b17 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:58:40.636927 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b2f501ce-5c9d-4697-b1ff-2de3bb188b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:40.739154 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f501ce-5c9d-4697-b1ff-2de3bb188b17 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:58:40.788102 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:40.812018 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa9d52-f00d-420e-b61d-0ec3ef7d0eec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:58:40.812612 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa9d52-f00d-420e-b61d-0ec3ef7d0eec service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:40.812914 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa9d52-f00d-420e-b61d-0ec3ef7d0eec service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:40.813178 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa9d52-f00d-420e-b61d-0ec3ef7d0eec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:40.814744 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-57aa9d52-f00d-420e-b61d-0ec3ef7d0eec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:40.920793 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa9d52-f00d-420e-b61d-0ec3ef7d0eec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:58:40.922230 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa9d52-f00d-420e-b61d-0ec3ef7d0eec service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:58:40.940274 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 566/1132] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:58:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:58:43.144960 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5d67b-6ba7-4a3e-8257-e37f6fb47b46 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:43.145497 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5d67b-6ba7-4a3e-8257-e37f6fb47b46 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:43.145632 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5d67b-6ba7-4a3e-8257-e37f6fb47b46 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:43.146874 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 567/1133] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 05:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:58:43.152262 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7873a7-a11a-41db-920f-a1bca878fc4e None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:43.152644 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7873a7-a11a-41db-920f-a1bca878fc4e None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:43.152868 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7873a7-a11a-41db-920f-a1bca878fc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:43.220816 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-ba7873a7-a11a-41db-920f-a1bca878fc4e None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:58:43.222012 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ba7873a7-a11a-41db-920f-a1bca878fc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:43.250993 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:43.319907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 567/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 05:58:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:58:43.610948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5d67b-6ba7-4a3e-8257-e37f6fb47b46 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:58:43.612539 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b4b5d67b-6ba7-4a3e-8257-e37f6fb47b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:43.719379 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5d67b-6ba7-4a3e-8257-e37f6fb47b46 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:58:43.746375 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:43.789911 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fbbc3d-b2b3-4aa8-9a95-c95f7aa54ab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:58:43.790501 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fbbc3d-b2b3-4aa8-9a95-c95f7aa54ab7 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:43.790773 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fbbc3d-b2b3-4aa8-9a95-c95f7aa54ab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:43.791037 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fbbc3d-b2b3-4aa8-9a95-c95f7aa54ab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:43.792605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-09fbbc3d-b2b3-4aa8-9a95-c95f7aa54ab7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:43.898470 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fbbc3d-b2b3-4aa8-9a95-c95f7aa54ab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:58:43.899880 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fbbc3d-b2b3-4aa8-9a95-c95f7aa54ab7 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:58:43.917667 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 568/1135] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:58:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:58:46.097271 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c7730-6c25-4818-98e2-1f8ec0403ee3 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:46.097816 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c7730-6c25-4818-98e2-1f8ec0403ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:46.097816 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c7730-6c25-4818-98e2-1f8ec0403ee3 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:46.099121 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 568/1136] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 05:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:58:46.104476 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8ee07-74ff-4bd0-9d39-a7d6827a4652 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:46.104938 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8ee07-74ff-4bd0-9d39-a7d6827a4652 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:46.105083 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8ee07-74ff-4bd0-9d39-a7d6827a4652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:46.171243 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-fee8ee07-74ff-4bd0-9d39-a7d6827a4652 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:58:46.172443 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-fee8ee07-74ff-4bd0-9d39-a7d6827a4652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:46.180649 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:46.268332 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 569/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 05:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:58:46.551606 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c7730-6c25-4818-98e2-1f8ec0403ee3 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:58:46.553223 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-bb6c7730-6c25-4818-98e2-1f8ec0403ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:46.655723 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c7730-6c25-4818-98e2-1f8ec0403ee3 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:58:46.711260 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:46.729337 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43055db4-1a80-48db-a537-06bb16d8b56e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:58:46.729938 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43055db4-1a80-48db-a537-06bb16d8b56e service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:46.730216 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43055db4-1a80-48db-a537-06bb16d8b56e service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:46.730485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43055db4-1a80-48db-a537-06bb16d8b56e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:46.733068 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-43055db4-1a80-48db-a537-06bb16d8b56e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:46.842602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43055db4-1a80-48db-a537-06bb16d8b56e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:58:46.843951 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43055db4-1a80-48db-a537-06bb16d8b56e service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:58:46.862356 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 569/1138] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:58:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:58:49.150290 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efae784-109a-46b4-91ac-9337a9fe86a7 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:49.150815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efae784-109a-46b4-91ac-9337a9fe86a7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:49.150815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efae784-109a-46b4-91ac-9337a9fe86a7 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:49.152156 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 570/1139] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 05:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:58:49.158176 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c653d4-865d-401d-b429-65def631890c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:49.158592 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c653d4-865d-401d-b429-65def631890c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:49.158735 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c653d4-865d-401d-b429-65def631890c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:49.224687 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-86c653d4-865d-401d-b429-65def631890c None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:58:49.225885 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-86c653d4-865d-401d-b429-65def631890c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:49.263965 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:49.322237 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 570/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 05:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:58:49.606184 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efae784-109a-46b4-91ac-9337a9fe86a7 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:58:49.607743 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9efae784-109a-46b4-91ac-9337a9fe86a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:49.712144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efae784-109a-46b4-91ac-9337a9fe86a7 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:58:49.748809 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:49.784030 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe265c-9c7c-4103-993a-70a719d176fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:58:49.784623 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe265c-9c7c-4103-993a-70a719d176fe service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:49.784924 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe265c-9c7c-4103-993a-70a719d176fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:49.785191 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe265c-9c7c-4103-993a-70a719d176fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:49.786755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-99fe265c-9c7c-4103-993a-70a719d176fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:49.891448 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fe265c-9c7c-4103-993a-70a719d176fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:58:49.892904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fe265c-9c7c-4103-993a-70a719d176fe service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:58:49.910266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 571/1141] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:58:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 05:58:52.065744 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97e51e-27d6-478d-b254-94f5b4f2b9df None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:52.066311 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97e51e-27d6-478d-b254-94f5b4f2b9df None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:52.066311 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97e51e-27d6-478d-b254-94f5b4f2b9df None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:52.067616 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 571/1142] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 05:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:58:52.072937 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc91e9-b50b-44bd-9c5e-45178d6916b9 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:52.073361 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc91e9-b50b-44bd-9c5e-45178d6916b9 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:52.073473 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc91e9-b50b-44bd-9c5e-45178d6916b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:52.139523 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-dbcc91e9-b50b-44bd-9c5e-45178d6916b9 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:58:52.140719 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-dbcc91e9-b50b-44bd-9c5e-45178d6916b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:52.158198 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:52.236151 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 572/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 05:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:58:52.525459 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97e51e-27d6-478d-b254-94f5b4f2b9df None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:58:52.527060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-4d97e51e-27d6-478d-b254-94f5b4f2b9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:52.631487 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97e51e-27d6-478d-b254-94f5b4f2b9df None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:58:52.699360 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:52.708019 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43723ade-5c5d-48a5-aa30-e9c0ce01c0a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:58:52.708619 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43723ade-5c5d-48a5-aa30-e9c0ce01c0a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:52.708934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43723ade-5c5d-48a5-aa30-e9c0ce01c0a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:52.709201 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43723ade-5c5d-48a5-aa30-e9c0ce01c0a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:52.710804 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-43723ade-5c5d-48a5-aa30-e9c0ce01c0a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:52.818260 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43723ade-5c5d-48a5-aa30-e9c0ce01c0a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:58:52.819602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43723ade-5c5d-48a5-aa30-e9c0ce01c0a3 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:58:52.837524 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 572/1144] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:58:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:58:55.045912 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481c11b-0b38-4487-b344-09c240598c77 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:55.046485 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481c11b-0b38-4487-b344-09c240598c77 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:55.046485 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481c11b-0b38-4487-b344-09c240598c77 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:55.047803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 573/1145] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 05:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:58:55.053211 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e2f05-1dc8-421b-96fd-d2d24752a189 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:55.053661 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e2f05-1dc8-421b-96fd-d2d24752a189 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:55.053773 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e2f05-1dc8-421b-96fd-d2d24752a189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:55.120627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-331e2f05-1dc8-421b-96fd-d2d24752a189 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:58:55.121888 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-331e2f05-1dc8-421b-96fd-d2d24752a189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:55.170357 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:55.217389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 573/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 05:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:58:55.504067 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a481c11b-0b38-4487-b344-09c240598c77 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:58:55.505717 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a481c11b-0b38-4487-b344-09c240598c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:55.614770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a481c11b-0b38-4487-b344-09c240598c77 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:58:55.665336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:58:55.694068 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405404bd-c9a3-4b44-a70d-54c890cfb48f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:58:55.694792 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405404bd-c9a3-4b44-a70d-54c890cfb48f service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:58:55.695062 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405404bd-c9a3-4b44-a70d-54c890cfb48f service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:58:55.695326 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405404bd-c9a3-4b44-a70d-54c890cfb48f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:58:55.696969 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-405404bd-c9a3-4b44-a70d-54c890cfb48f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:58:55.809278 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405404bd-c9a3-4b44-a70d-54c890cfb48f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:58:55.811100 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405404bd-c9a3-4b44-a70d-54c890cfb48f service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:58:55.829440 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 574/1147] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:58:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:59:05.215311 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=22 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:59:05.827182 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=66 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:59:15.849181 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce56112-caf0-4cf6-bbf5-825c678f3121 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:59:15.849746 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce56112-caf0-4cf6-bbf5-825c678f3121 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:59:15.849746 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce56112-caf0-4cf6-bbf5-825c678f3121 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:59:15.851025 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 574/1148] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:59:15.857113 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dd3ff-0ff7-4ed2-8146-24e9ae64b310 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:59:15.857549 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dd3ff-0ff7-4ed2-8146-24e9ae64b310 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:59:15.857639 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dd3ff-0ff7-4ed2-8146-24e9ae64b310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:59:15.922291 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-000dd3ff-0ff7-4ed2-8146-24e9ae64b310 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:59:15.923393 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-000dd3ff-0ff7-4ed2-8146-24e9ae64b310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:59:16.018193 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 575/1149] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:59:16.324231 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaa8e0-9d0d-42b7-8135-6baec2dbc77c None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:59:16.324520 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaa8e0-9d0d-42b7-8135-6baec2dbc77c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:59:16.324834 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaa8e0-9d0d-42b7-8135-6baec2dbc77c None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:59:16.326196 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 575/1150] 10.222.0.20 () {54 vars in 962 bytes} [Thu Oct 20 05:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:59:16.331159 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059ed53-e0e3-4042-99cc-a8cf8c47f0fc None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:59:16.331445 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059ed53-e0e3-4042-99cc-a8cf8c47f0fc None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:59:16.331720 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059ed53-e0e3-4042-99cc-a8cf8c47f0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:59:16.408296 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-7059ed53-e0e3-4042-99cc-a8cf8c47f0fc None None] MFA Rules not processed for user `3dcfe5244bd04c00b6dde1d6f795d4ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:59:16.409509 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7059ed53-e0e3-4042-99cc-a8cf8c47f0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:59:16.429851 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:59:16.483677 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 576/1151] 10.222.0.20 () {58 vars in 1059 bytes} [Thu Oct 20 05:59:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:59:16.491467 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccaa8e0-9d0d-42b7-8135-6baec2dbc77c None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:59:16.493016 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6ccaa8e0-9d0d-42b7-8135-6baec2dbc77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:59:16.595228 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccaa8e0-9d0d-42b7-8135-6baec2dbc77c None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:59:16.667374 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d464fd4-82e0-470c-8803-d07af36be9e9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dcfe5244bd04c00b6dde1d6f795d4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:59:16.667972 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d464fd4-82e0-470c-8803-d07af36be9e9 service glance] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:59:16.668249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d464fd4-82e0-470c-8803-d07af36be9e9 service glance] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:59:16.668518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d464fd4-82e0-470c-8803-d07af36be9e9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:59:16.670251 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1d464fd4-82e0-470c-8803-d07af36be9e9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:59:16.722616 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:59:16.777306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d464fd4-82e0-470c-8803-d07af36be9e9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:59:16.779514 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d464fd4-82e0-470c-8803-d07af36be9e9 service glance] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:59:16.797536 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 576/1152] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:59:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:59:19.018127 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec674b78-73f6-4cf1-9beb-00aef6246bfc None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:59:19.018578 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec674b78-73f6-4cf1-9beb-00aef6246bfc None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:59:19.018704 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec674b78-73f6-4cf1-9beb-00aef6246bfc None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:59:19.020037 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 577/1153] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:59:19.025478 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd88714-85b7-4ee8-a11a-a4b46b09e2a2 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:59:19.025905 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd88714-85b7-4ee8-a11a-a4b46b09e2a2 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:59:19.026030 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd88714-85b7-4ee8-a11a-a4b46b09e2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:59:19.090284 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-9dd88714-85b7-4ee8-a11a-a4b46b09e2a2 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:59:19.092159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-9dd88714-85b7-4ee8-a11a-a4b46b09e2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:59:19.187952 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 577/1154] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 05:59:19.461950 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec674b78-73f6-4cf1-9beb-00aef6246bfc None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:59:19.463515 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ec674b78-73f6-4cf1-9beb-00aef6246bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:59:19.588201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec674b78-73f6-4cf1-9beb-00aef6246bfc None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:59:19.636661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:59:19.680493 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6ec42-dcac-4781-9e43-c60047df80ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:59:19.681113 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6ec42-dcac-4781-9e43-c60047df80ba admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:59:19.681387 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6ec42-dcac-4781-9e43-c60047df80ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:59:19.681655 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6ec42-dcac-4781-9e43-c60047df80ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:59:19.682271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6ec42-dcac-4781-9e43-c60047df80ba admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:59:19.683366 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: WARNING py.warnings [None req-52a6ec42-dcac-4781-9e43-c60047df80ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 05:59:19.683366 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: warnings.warn(msg) Oct 20 05:59:19.683366 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: Oct 20 05:59:19.684536 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6ec42-dcac-4781-9e43-c60047df80ba admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:59:19.703448 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 578/1155] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Oct 20 05:59:19 2022] GET /identity/v3/projects => generated 2553 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 05:59:21.898486 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86638b-219e-4249-bb4f-27fc6148c658 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:59:21.899049 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86638b-219e-4249-bb4f-27fc6148c658 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:59:21.899049 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86638b-219e-4249-bb4f-27fc6148c658 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:59:21.900393 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 578/1156] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:59:21.905722 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0998bd-5c16-4ed1-bff7-d29778f70d6e None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:59:21.906119 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0998bd-5c16-4ed1-bff7-d29778f70d6e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:59:21.906306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0998bd-5c16-4ed1-bff7-d29778f70d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:59:21.972103 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-7b0998bd-5c16-4ed1-bff7-d29778f70d6e None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:59:21.973343 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7b0998bd-5c16-4ed1-bff7-d29778f70d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:59:22.069597 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 579/1157] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:59:22.360492 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86638b-219e-4249-bb4f-27fc6148c658 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:59:22.362138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6c86638b-219e-4249-bb4f-27fc6148c658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:59:22.371968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:59:22.508269 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86638b-219e-4249-bb4f-27fc6148c658 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:59:22.588534 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:59:22.621221 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9708071-c99f-48fd-8054-eea0fb587095 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:59:22.621828 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9708071-c99f-48fd-8054-eea0fb587095 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:59:22.622255 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9708071-c99f-48fd-8054-eea0fb587095 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:59:22.622620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9708071-c99f-48fd-8054-eea0fb587095 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:59:22.624235 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e9708071-c99f-48fd-8054-eea0fb587095 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:59:22.712548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9708071-c99f-48fd-8054-eea0fb587095 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:59:22.714161 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9708071-c99f-48fd-8054-eea0fb587095 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:59:22.732392 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 579/1158] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:59:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:59:25.067298 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffa526-ad17-4c5c-b977-ce192ec5d101 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:59:25.067866 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffa526-ad17-4c5c-b977-ce192ec5d101 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:59:25.067866 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffa526-ad17-4c5c-b977-ce192ec5d101 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:59:25.069231 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 580/1159] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:59:25.074518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2a6ac-c827-4393-93e6-9b8081969dfe None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:59:25.074943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2a6ac-c827-4393-93e6-9b8081969dfe None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:59:25.075124 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2a6ac-c827-4393-93e6-9b8081969dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:59:25.139620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-46e2a6ac-c827-4393-93e6-9b8081969dfe None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:59:25.140840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-46e2a6ac-c827-4393-93e6-9b8081969dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:59:25.189175 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:59:25.238892 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 580/1160] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:59:25.536213 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ffa526-ad17-4c5c-b977-ce192ec5d101 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:59:25.537796 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d2ffa526-ad17-4c5c-b977-ce192ec5d101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:59:25.561471 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:59:25.641776 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ffa526-ad17-4c5c-b977-ce192ec5d101 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:59:25.713789 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873632b-3900-4cf5-92fe-ef6617305020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:59:25.714393 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873632b-3900-4cf5-92fe-ef6617305020 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:59:25.714676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873632b-3900-4cf5-92fe-ef6617305020 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:59:25.714941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873632b-3900-4cf5-92fe-ef6617305020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:59:25.717182 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1873632b-3900-4cf5-92fe-ef6617305020 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:59:25.790207 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:59:25.824689 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873632b-3900-4cf5-92fe-ef6617305020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:59:25.826974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873632b-3900-4cf5-92fe-ef6617305020 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:59:25.849080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 581/1161] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:59:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:59:28.055148 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e63724-b79c-4ca0-9107-28f873ec7a52 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:59:28.055723 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e63724-b79c-4ca0-9107-28f873ec7a52 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:59:28.055723 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e63724-b79c-4ca0-9107-28f873ec7a52 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:59:28.057066 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 581/1162] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:59:28.062311 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1862b7-44ee-40da-99b7-55d41d130ad4 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:59:28.062740 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1862b7-44ee-40da-99b7-55d41d130ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:59:28.062861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1862b7-44ee-40da-99b7-55d41d130ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:59:28.128978 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-db1862b7-44ee-40da-99b7-55d41d130ad4 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:59:28.130196 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-db1862b7-44ee-40da-99b7-55d41d130ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:59:28.228260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 582/1163] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:59:28.518434 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e63724-b79c-4ca0-9107-28f873ec7a52 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:59:28.519970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e1e63724-b79c-4ca0-9107-28f873ec7a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:59:28.622022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e63724-b79c-4ca0-9107-28f873ec7a52 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:59:28.693672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:59:28.695147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f8955-7482-4c6c-976b-9e30550d73b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:59:28.695841 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f8955-7482-4c6c-976b-9e30550d73b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:59:28.696150 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f8955-7482-4c6c-976b-9e30550d73b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:59:28.696453 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f8955-7482-4c6c-976b-9e30550d73b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:59:28.698641 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-772f8955-7482-4c6c-976b-9e30550d73b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:59:28.806372 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f8955-7482-4c6c-976b-9e30550d73b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:59:28.807716 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f8955-7482-4c6c-976b-9e30550d73b5 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:59:28.825617 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 582/1164] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:59:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:59:31.079724 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaa229-8c27-440d-9449-89566226a218 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:59:31.080327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaa229-8c27-440d-9449-89566226a218 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:59:31.080327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaa229-8c27-440d-9449-89566226a218 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:59:31.081657 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 583/1165] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:59:31.086940 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa70909-493b-4a23-a225-fea5edca23cb None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:59:31.087396 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa70909-493b-4a23-a225-fea5edca23cb None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:59:31.087508 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa70909-493b-4a23-a225-fea5edca23cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:59:31.153189 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-eaa70909-493b-4a23-a225-fea5edca23cb None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:59:31.155165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-eaa70909-493b-4a23-a225-fea5edca23cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:59:31.210037 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:59:31.255820 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 583/1166] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:59:31.550667 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aaa229-8c27-440d-9449-89566226a218 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:59:31.552209 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-47aaa229-8c27-440d-9449-89566226a218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:59:31.589482 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:59:31.658419 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aaa229-8c27-440d-9449-89566226a218 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:59:31.729774 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68913cd-78f4-4d7a-b19e-412622f9a14a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:59:31.730372 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68913cd-78f4-4d7a-b19e-412622f9a14a service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:59:31.730650 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68913cd-78f4-4d7a-b19e-412622f9a14a service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:59:31.730913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68913cd-78f4-4d7a-b19e-412622f9a14a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:59:31.733016 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a68913cd-78f4-4d7a-b19e-412622f9a14a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:59:31.812201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:59:31.840415 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68913cd-78f4-4d7a-b19e-412622f9a14a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:59:31.842413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68913cd-78f4-4d7a-b19e-412622f9a14a service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:59:31.860414 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 584/1167] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:59:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:59:34.246995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e562d2e-6faa-4db8-84ac-9c1843e8ca06 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:59:34.248032 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e562d2e-6faa-4db8-84ac-9c1843e8ca06 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:59:34.248032 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e562d2e-6faa-4db8-84ac-9c1843e8ca06 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:59:34.249422 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 584/1168] 10.222.0.20 () {54 vars in 927 bytes} [Thu Oct 20 05:59:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 05:59:34.254883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667c533-1298-4b23-b109-d0d1f7e66411 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:59:34.255267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667c533-1298-4b23-b109-d0d1f7e66411 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:59:34.255446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667c533-1298-4b23-b109-d0d1f7e66411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:59:34.324358 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-4667c533-1298-4b23-b109-d0d1f7e66411 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 05:59:34.326197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4667c533-1298-4b23-b109-d0d1f7e66411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:59:34.418841 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 585/1169] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Oct 20 05:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 05:59:34.708847 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e562d2e-6faa-4db8-84ac-9c1843e8ca06 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 05:59:34.710417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5e562d2e-6faa-4db8-84ac-9c1843e8ca06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:59:34.812766 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e562d2e-6faa-4db8-84ac-9c1843e8ca06 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 05:59:34.885259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770113c-27dd-4b48-a092-7c939a6921c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 05:59:34.885861 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770113c-27dd-4b48-a092-7c939a6921c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 05:59:34.886137 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770113c-27dd-4b48-a092-7c939a6921c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 05:59:34.886401 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770113c-27dd-4b48-a092-7c939a6921c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 05:59:34.888118 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8770113c-27dd-4b48-a092-7c939a6921c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 05:59:34.897506 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:59:34.998613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770113c-27dd-4b48-a092-7c939a6921c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 05:59:34.999968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770113c-27dd-4b48-a092-7c939a6921c2 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 05:59:35.018047 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 585/1170] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 05:59:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 05:59:44.417274 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=82 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 05:59:45.015749 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=50 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:00:47.932428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a7aed-704e-41ac-83de-671ec0b2c212 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:00:47.933044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a7aed-704e-41ac-83de-671ec0b2c212 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:00:47.933044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a7aed-704e-41ac-83de-671ec0b2c212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:00:48.010558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-522a7aed-704e-41ac-83de-671ec0b2c212 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:00:48.011694 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-522a7aed-704e-41ac-83de-671ec0b2c212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:00:48.105592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 586/1171] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:00:48.113788 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165515a-a045-465f-9cac-6acb996297a0 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:00:48.114279 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165515a-a045-465f-9cac-6acb996297a0 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:00:48.114381 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165515a-a045-465f-9cac-6acb996297a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:00:48.192537 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3165515a-a045-465f-9cac-6acb996297a0 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:00:48.193866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3165515a-a045-465f-9cac-6acb996297a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:00:48.291257 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 586/1172] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:00:48.295812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a7aed-704e-41ac-83de-671ec0b2c212 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:00:48.297159 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-522a7aed-704e-41ac-83de-671ec0b2c212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:00:48.308687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a7aed-704e-41ac-83de-671ec0b2c212 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:00:48.369046 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:00:48.403203 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85bee4-f69b-442d-afa5-61c52dc9e40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:00:48.403804 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85bee4-f69b-442d-afa5-61c52dc9e40d admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:00:48.404112 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85bee4-f69b-442d-afa5-61c52dc9e40d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:00:48.404375 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85bee4-f69b-442d-afa5-61c52dc9e40d admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:00:48.405049 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85bee4-f69b-442d-afa5-61c52dc9e40d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:00:48.405351 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85bee4-f69b-442d-afa5-61c52dc9e40d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:00:48.406601 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: WARNING py.warnings [None req-6e85bee4-f69b-442d-afa5-61c52dc9e40d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 20 06:00:48.406601 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: warnings.warn(msg) Oct 20 06:00:48.406601 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: Oct 20 06:00:48.407509 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85bee4-f69b-442d-afa5-61c52dc9e40d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:00:48.418623 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 587/1173] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:00:48.428086 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165515a-a045-465f-9cac-6acb996297a0 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:00:48.437241 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165515a-a045-465f-9cac-6acb996297a0 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:00:48.499475 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:00:48.533572 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87eb9d-5799-448e-aa58-1eeed2cc3a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:00:48.534175 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87eb9d-5799-448e-aa58-1eeed2cc3a70 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:00:48.534452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87eb9d-5799-448e-aa58-1eeed2cc3a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:00:48.534720 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87eb9d-5799-448e-aa58-1eeed2cc3a70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:00:48.535347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87eb9d-5799-448e-aa58-1eeed2cc3a70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:00:48.536359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: WARNING py.warnings [None req-1b87eb9d-5799-448e-aa58-1eeed2cc3a70 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 06:00:48.536359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: warnings.warn(msg) Oct 20 06:00:48.536359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: Oct 20 06:00:48.537565 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87eb9d-5799-448e-aa58-1eeed2cc3a70 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:00:48.592504 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 587/1174] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:00:48 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:00:48.597037 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85bee4-f69b-442d-afa5-61c52dc9e40d admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:00:48.598586 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6e85bee4-f69b-442d-afa5-61c52dc9e40d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:00:48.631926 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85bee4-f69b-442d-afa5-61c52dc9e40d admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:00:48.722545 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17610b8-0f79-4f92-9ba5-29c253cf1f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:00:48.723136 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17610b8-0f79-4f92-9ba5-29c253cf1f06 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:00:48.723408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17610b8-0f79-4f92-9ba5-29c253cf1f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:00:48.723672 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17610b8-0f79-4f92-9ba5-29c253cf1f06 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:00:48.724283 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17610b8-0f79-4f92-9ba5-29c253cf1f06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:00:48.725399 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: WARNING py.warnings [None req-a17610b8-0f79-4f92-9ba5-29c253cf1f06 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 06:00:48.725399 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: warnings.warn(msg) Oct 20 06:00:48.725399 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: Oct 20 06:00:48.726594 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17610b8-0f79-4f92-9ba5-29c253cf1f06 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:00:48.768445 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 588/1175] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:00:48 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:00:48.772508 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87eb9d-5799-448e-aa58-1eeed2cc3a70 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:00:48.774090 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1b87eb9d-5799-448e-aa58-1eeed2cc3a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:00:48.835137 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87eb9d-5799-448e-aa58-1eeed2cc3a70 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:00:48.915981 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:00:48.928120 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce595f-ed1f-4bc8-8ffe-5ab5031f07ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:00:48.928812 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce595f-ed1f-4bc8-8ffe-5ab5031f07ca admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:00:48.929144 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce595f-ed1f-4bc8-8ffe-5ab5031f07ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:00:48.929511 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce595f-ed1f-4bc8-8ffe-5ab5031f07ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:00:48.930181 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce595f-ed1f-4bc8-8ffe-5ab5031f07ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:00:48.931800 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce595f-ed1f-4bc8-8ffe-5ab5031f07ca admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:00:48.944763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 588/1176] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:00:48.948961 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17610b8-0f79-4f92-9ba5-29c253cf1f06 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:00:48.950536 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a17610b8-0f79-4f92-9ba5-29c253cf1f06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:00:48.961706 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17610b8-0f79-4f92-9ba5-29c253cf1f06 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:00:48.986954 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:00:49.058247 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf3f68-4433-45df-abef-3a63cc25a7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:00:49.058849 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf3f68-4433-45df-abef-3a63cc25a7da admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:00:49.059121 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf3f68-4433-45df-abef-3a63cc25a7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:00:49.059385 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf3f68-4433-45df-abef-3a63cc25a7da admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:00:49.060012 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf3f68-4433-45df-abef-3a63cc25a7da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:00:49.061624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf3f68-4433-45df-abef-3a63cc25a7da admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:00:49.073529 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 589/1177] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:00:49.077706 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce595f-ed1f-4bc8-8ffe-5ab5031f07ca admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:00:49.079418 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7dce595f-ed1f-4bc8-8ffe-5ab5031f07ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:00:49.113195 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce595f-ed1f-4bc8-8ffe-5ab5031f07ca admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:00:49.230315 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdcdb12-26c6-4368-9523-b896c7684af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:00:49.231009 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcdb12-26c6-4368-9523-b896c7684af9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:00:49.231151 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcdb12-26c6-4368-9523-b896c7684af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:00:49.231421 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcdb12-26c6-4368-9523-b896c7684af9 admin admin] PATH_INFO: `/v3/projects/4dcf5b9c95b846fca275044c56cfcf4b/users/a5f0da0c6b784472a4657f25ff775ae5/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:00:49.254528 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdcdb12-26c6-4368-9523-b896c7684af9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dcf5b9c95b846fca275044c56cfcf4b, user_id=a5f0da0c6b784472a4657f25ff775ae5, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:00:49.257545 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdcdb12-26c6-4368-9523-b896c7684af9 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:00:49.282563 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 589/1178] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:00:49 2022] PUT /identity/v3/projects/4dcf5b9c95b846fca275044c56cfcf4b/users/a5f0da0c6b784472a4657f25ff775ae5/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:00:49.293898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bca02-4b0d-4932-a6da-e80aac84992b None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:00:49.294183 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bca02-4b0d-4932-a6da-e80aac84992b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:00:49.294458 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bca02-4b0d-4932-a6da-e80aac84992b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:00:49.372714 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-ff3bca02-4b0d-4932-a6da-e80aac84992b None None] MFA Rules not processed for user `a5f0da0c6b784472a4657f25ff775ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:00:49.374141 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ff3bca02-4b0d-4932-a6da-e80aac84992b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:00:49.465984 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 590/1179] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:00:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:00:49.479752 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e593d34-f425-4e5d-8848-d3fc6a4e32fb None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:00:49.480038 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e593d34-f425-4e5d-8848-d3fc6a4e32fb None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:00:49.480312 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e593d34-f425-4e5d-8848-d3fc6a4e32fb None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:00:49.481766 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 590/1180] 10.222.0.20 () {54 vars in 963 bytes} [Thu Oct 20 06:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 06:00:49.487437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9eadf1-95d1-4dd5-95c3-a981fff7985d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:00:49.487889 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9eadf1-95d1-4dd5-95c3-a981fff7985d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:00:49.488360 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9eadf1-95d1-4dd5-95c3-a981fff7985d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:00:49.528886 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:00:49.579411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-bc9eadf1-95d1-4dd5-95c3-a981fff7985d None None] MFA Rules not processed for user `cfb00a74a99e47c1ac938f07d26a4bc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:00:49.580632 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-bc9eadf1-95d1-4dd5-95c3-a981fff7985d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:00:49.659630 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 591/1181] 10.222.0.20 () {58 vars in 1060 bytes} [Thu Oct 20 06:00:49 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:00:49.667662 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e593d34-f425-4e5d-8848-d3fc6a4e32fb None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:00:49.669268 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-4e593d34-f425-4e5d-8848-d3fc6a4e32fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:00:49.726247 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:00:49.790940 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e593d34-f425-4e5d-8848-d3fc6a4e32fb None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:00:49.876617 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccaa91c-786f-4650-ba8a-c60aaf43c7cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:00:49.877257 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccaa91c-786f-4650-ba8a-c60aaf43c7cb service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:00:49.877542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccaa91c-786f-4650-ba8a-c60aaf43c7cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:00:49.877812 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccaa91c-786f-4650-ba8a-c60aaf43c7cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:00:49.879442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-bccaa91c-786f-4650-ba8a-c60aaf43c7cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:00:49.983529 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccaa91c-786f-4650-ba8a-c60aaf43c7cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:00:49.985630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccaa91c-786f-4650-ba8a-c60aaf43c7cb service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:00:49.989172 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:00:50.003941 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 591/1182] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:00:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:00:50.049880 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f188ad1-0fbe-47f9-93c3-589055f9ce04 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:00:50.050324 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f188ad1-0fbe-47f9-93c3-589055f9ce04 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:00:50.050431 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f188ad1-0fbe-47f9-93c3-589055f9ce04 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:00:50.051691 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 592/1183] 10.222.0.20 () {52 vars in 830 bytes} [Thu Oct 20 06:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 06:00:50.055268 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccaa91c-786f-4650-ba8a-c60aaf43c7cb service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:00:50.056468 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-bccaa91c-786f-4650-ba8a-c60aaf43c7cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:00:50.159557 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccaa91c-786f-4650-ba8a-c60aaf43c7cb service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:00:50.256352 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014aaba-5fde-4384-b3b2-e1e4881f09ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:00:50.257314 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014aaba-5fde-4384-b3b2-e1e4881f09ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:00:50.257540 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014aaba-5fde-4384-b3b2-e1e4881f09ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:00:50.257814 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014aaba-5fde-4384-b3b2-e1e4881f09ab admin admin] PATH_INFO: `/v3/users/a5f0da0c6b784472a4657f25ff775ae5` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:00:50.271376 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014aaba-5fde-4384-b3b2-e1e4881f09ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5f0da0c6b784472a4657f25ff775ae5)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:00:50.272397 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: WARNING py.warnings [None req-c014aaba-5fde-4384-b3b2-e1e4881f09ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 06:00:50.272397 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: warnings.warn(msg) Oct 20 06:00:50.272397 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: Oct 20 06:00:50.273666 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014aaba-5fde-4384-b3b2-e1e4881f09ab admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:00:50.283700 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:00:50.338724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c014aaba-5fde-4384-b3b2-e1e4881f09ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5f0da0c6b784472a4657f25ff775ae5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:00:50.339253 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c014aaba-5fde-4384-b3b2-e1e4881f09ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5f0da0c6b784472a4657f25ff775ae5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:00:50.349785 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c014aaba-5fde-4384-b3b2-e1e4881f09ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5f0da0c6b784472a4657f25ff775ae5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:00:50.373549 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c014aaba-5fde-4384-b3b2-e1e4881f09ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5f0da0c6b784472a4657f25ff775ae5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:00:50.387512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c014aaba-5fde-4384-b3b2-e1e4881f09ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5f0da0c6b784472a4657f25ff775ae5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:00:50.393520 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 592/1184] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:00:50 2022] DELETE /identity/v3/users/a5f0da0c6b784472a4657f25ff775ae5 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:00:50.398274 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be3d9e-6fb6-4ddb-8cd2-95caefc02eb0 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:00:50.398572 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be3d9e-6fb6-4ddb-8cd2-95caefc02eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:00:50.398865 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be3d9e-6fb6-4ddb-8cd2-95caefc02eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:00:50.463665 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-07be3d9e-6fb6-4ddb-8cd2-95caefc02eb0 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:00:50.464933 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-07be3d9e-6fb6-4ddb-8cd2-95caefc02eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:00:50.539551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:00:50.553424 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 593/1185] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:00:50.563801 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014aaba-5fde-4384-b3b2-e1e4881f09ab admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:00:50.565053 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c014aaba-5fde-4384-b3b2-e1e4881f09ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:00:50.664108 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014aaba-5fde-4384-b3b2-e1e4881f09ab admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:00:50.716871 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:00:50.737283 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df119f-6dfd-43e4-b9c6-be4f78b7b2fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:00:50.737892 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df119f-6dfd-43e4-b9c6-be4f78b7b2fa service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:00:50.738170 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df119f-6dfd-43e4-b9c6-be4f78b7b2fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:00:50.738440 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df119f-6dfd-43e4-b9c6-be4f78b7b2fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:00:50.740021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-35df119f-6dfd-43e4-b9c6-be4f78b7b2fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:00:50.847587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df119f-6dfd-43e4-b9c6-be4f78b7b2fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:00:50.848968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df119f-6dfd-43e4-b9c6-be4f78b7b2fa service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:00:50.866461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 593/1186] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:00:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:00:51.190258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be3d9e-6fb6-4ddb-8cd2-95caefc02eb0 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:00:51.191535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-07be3d9e-6fb6-4ddb-8cd2-95caefc02eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:00:51.203147 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be3d9e-6fb6-4ddb-8cd2-95caefc02eb0 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:00:51.303353 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c363b-3014-4038-911a-1ccad4e3aa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:00:51.303353 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c363b-3014-4038-911a-1ccad4e3aa98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:00:51.303353 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c363b-3014-4038-911a-1ccad4e3aa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:00:51.303353 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c363b-3014-4038-911a-1ccad4e3aa98 admin admin] PATH_INFO: `/v3/projects/4dcf5b9c95b846fca275044c56cfcf4b` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:00:51.313357 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c363b-3014-4038-911a-1ccad4e3aa98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dcf5b9c95b846fca275044c56cfcf4b)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:00:51.314384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: WARNING py.warnings [None req-511c363b-3014-4038-911a-1ccad4e3aa98 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 06:00:51.314384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: warnings.warn(msg) Oct 20 06:00:51.314384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: Oct 20 06:00:51.315572 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c363b-3014-4038-911a-1ccad4e3aa98 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:00:51.372250 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-511c363b-3014-4038-911a-1ccad4e3aa98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dcf5b9c95b846fca275044c56cfcf4b', 'request_id': 'req-511c363b-3014-4038-911a-1ccad4e3aa98'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:00:51.378167 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-511c363b-3014-4038-911a-1ccad4e3aa98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dcf5b9c95b846fca275044c56cfcf4b', 'request_id': 'req-511c363b-3014-4038-911a-1ccad4e3aa98'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:00:51.391604 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-511c363b-3014-4038-911a-1ccad4e3aa98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dcf5b9c95b846fca275044c56cfcf4b', 'request_id': 'req-511c363b-3014-4038-911a-1ccad4e3aa98'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:00:51.418045 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-511c363b-3014-4038-911a-1ccad4e3aa98 admin admin] The token cache is being invalidate because project 4dcf5b9c95b846fca275044c56cfcf4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:00:51.418341 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-511c363b-3014-4038-911a-1ccad4e3aa98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:00:51.418855 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-511c363b-3014-4038-911a-1ccad4e3aa98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:00:51.420151 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 594/1187] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:00:51 2022] DELETE /identity/v3/projects/4dcf5b9c95b846fca275044c56cfcf4b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:01:00.864201 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=62 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:01:01.416004 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:02:29.103972 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d45636-f70e-4caa-99c8-eeda67934f84 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:29.104577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d45636-f70e-4caa-99c8-eeda67934f84 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:29.104577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d45636-f70e-4caa-99c8-eeda67934f84 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:29.105929 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 594/1188] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 06:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 06:02:29.110690 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6284d-5bd0-41cf-afc1-80a0cc61c1b0 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:29.111104 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6284d-5bd0-41cf-afc1-80a0cc61c1b0 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:29.111234 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6284d-5bd0-41cf-afc1-80a0cc61c1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:29.192373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-03f6284d-5bd0-41cf-afc1-80a0cc61c1b0 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:02:29.193748 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-03f6284d-5bd0-41cf-afc1-80a0cc61c1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:02:29.296099 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 595/1189] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 06:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:02:29.450855 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d45636-f70e-4caa-99c8-eeda67934f84 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:02:29.453022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-13d45636-f70e-4caa-99c8-eeda67934f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:02:29.572715 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d45636-f70e-4caa-99c8-eeda67934f84 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:02:29.578175 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506bc4bc-74cd-4113-9ba0-04d7173a40cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:02:29.578780 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bc4bc-74cd-4113-9ba0-04d7173a40cf service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:29.579057 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bc4bc-74cd-4113-9ba0-04d7173a40cf service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:29.579359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bc4bc-74cd-4113-9ba0-04d7173a40cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:29.587585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506bc4bc-74cd-4113-9ba0-04d7173a40cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:02:29.589220 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506bc4bc-74cd-4113-9ba0-04d7173a40cf service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:02:29.599344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 595/1190] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:02:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:02:30.688896 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b8128-6490-4703-a390-5233f5133649 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:30.689455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b8128-6490-4703-a390-5233f5133649 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:30.689455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b8128-6490-4703-a390-5233f5133649 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:30.690779 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 596/1191] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 06:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 06:02:30.697130 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b28b25-533b-47d1-93b8-124f8da98ddc None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:30.697552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b28b25-533b-47d1-93b8-124f8da98ddc None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:30.697658 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b28b25-533b-47d1-93b8-124f8da98ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:30.718972 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-33b28b25-533b-47d1-93b8-124f8da98ddc None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:02:30.720166 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-33b28b25-533b-47d1-93b8-124f8da98ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:02:30.728653 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 596/1192] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 06:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:02:30.880769 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b8128-6490-4703-a390-5233f5133649 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:02:30.895558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b8128-6490-4703-a390-5233f5133649 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:02:30.901182 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7551e5-7f1a-424a-b8ec-6843547a05ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:02:30.901784 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7551e5-7f1a-424a-b8ec-6843547a05ba service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:30.902063 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7551e5-7f1a-424a-b8ec-6843547a05ba service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:30.902335 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7551e5-7f1a-424a-b8ec-6843547a05ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:30.910544 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7551e5-7f1a-424a-b8ec-6843547a05ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:02:30.912310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7551e5-7f1a-424a-b8ec-6843547a05ba service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:02:30.921536 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 597/1193] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:02:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:02:31.229866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdafdd1-b7e0-4954-918e-9c24f53d42e7 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:31.230277 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdafdd1-b7e0-4954-918e-9c24f53d42e7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:31.230552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdafdd1-b7e0-4954-918e-9c24f53d42e7 None None] PATH_INFO: `/v3` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:31.231827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 597/1194] 10.222.0.18 () {54 vars in 925 bytes} [Thu Oct 20 06:02:31 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:02:31.248476 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469abd76-8071-4084-b17f-33b168c88928 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:31.248796 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469abd76-8071-4084-b17f-33b168c88928 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:31.249074 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469abd76-8071-4084-b17f-33b168c88928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:31.310861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-469abd76-8071-4084-b17f-33b168c88928 None None] MFA Rules not processed for user `cfbaaf32475a4c82a34948602d8f4e35`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:02:31.312067 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-469abd76-8071-4084-b17f-33b168c88928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:02:31.328976 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:02:31.382514 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 598/1195] 10.222.0.18 () {58 vars in 1015 bytes} [Thu Oct 20 06:02:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:02:34.987114 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b1c2e-2729-4364-bf4e-432044cab303 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:34.987636 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b1c2e-2729-4364-bf4e-432044cab303 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:34.987636 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b1c2e-2729-4364-bf4e-432044cab303 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:34.988987 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 598/1196] 10.222.0.20 () {54 vars in 961 bytes} [Thu Oct 20 06:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 06:02:34.996371 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b84ff5-5efd-4483-b98a-d6e6b59b3039 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:34.996886 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b84ff5-5efd-4483-b98a-d6e6b59b3039 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:34.998030 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b84ff5-5efd-4483-b98a-d6e6b59b3039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:35.018873 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-10b84ff5-5efd-4483-b98a-d6e6b59b3039 None None] MFA Rules not processed for user `cfbaaf32475a4c82a34948602d8f4e35`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:02:35.020059 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-10b84ff5-5efd-4483-b98a-d6e6b59b3039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:02:35.029064 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 599/1197] 10.222.0.20 () {58 vars in 1058 bytes} [Thu Oct 20 06:02:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:02:35.037577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b1c2e-2729-4364-bf4e-432044cab303 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:02:35.050596 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b1c2e-2729-4364-bf4e-432044cab303 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:02:35.055469 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e15011-dd57-42ba-acb6-b26fc49c0e74 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfbaaf32475a4c82a34948602d8f4e35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:02:35.056077 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e15011-dd57-42ba-acb6-b26fc49c0e74 service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:35.056353 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e15011-dd57-42ba-acb6-b26fc49c0e74 service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:35.056618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e15011-dd57-42ba-acb6-b26fc49c0e74 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:35.064600 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e15011-dd57-42ba-acb6-b26fc49c0e74 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:02:35.066233 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e15011-dd57-42ba-acb6-b26fc49c0e74 service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:02:35.075599 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 599/1198] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:02:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:02:42.393595 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1fb9b-cea9-49b9-b238-f1d3e8e64e5c None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:42.394118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1fb9b-cea9-49b9-b238-f1d3e8e64e5c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:42.394118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1fb9b-cea9-49b9-b238-f1d3e8e64e5c None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:42.395458 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 600/1199] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 06:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 06:02:42.402836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bff040-b302-4492-8e8f-49a85b85f92c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:42.403268 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bff040-b302-4492-8e8f-49a85b85f92c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:42.403434 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bff040-b302-4492-8e8f-49a85b85f92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:42.423132 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-25bff040-b302-4492-8e8f-49a85b85f92c None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:02:42.424338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-25bff040-b302-4492-8e8f-49a85b85f92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:02:42.432248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 600/1200] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 06:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:02:42.588037 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1fb9b-cea9-49b9-b238-f1d3e8e64e5c None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:02:42.603248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1fb9b-cea9-49b9-b238-f1d3e8e64e5c None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:02:42.608838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d6d39-d8a2-4a1e-8210-05d6fe091a53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:02:42.609426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d6d39-d8a2-4a1e-8210-05d6fe091a53 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:42.609699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d6d39-d8a2-4a1e-8210-05d6fe091a53 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:42.609967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d6d39-d8a2-4a1e-8210-05d6fe091a53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:42.617993 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d6d39-d8a2-4a1e-8210-05d6fe091a53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:02:42.619548 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d6d39-d8a2-4a1e-8210-05d6fe091a53 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:02:42.628843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 601/1201] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:02:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:02:43.696682 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a4f22-e347-4d1a-a8ba-3d3a3f9aa166 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:43.697300 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a4f22-e347-4d1a-a8ba-3d3a3f9aa166 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:43.697300 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a4f22-e347-4d1a-a8ba-3d3a3f9aa166 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:43.698622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 601/1202] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 06:02:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 06:02:43.704630 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bb2dc-0a35-4821-8129-218c2747cc54 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:43.705119 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bb2dc-0a35-4821-8129-218c2747cc54 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:43.705216 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bb2dc-0a35-4821-8129-218c2747cc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:43.724722 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-435bb2dc-0a35-4821-8129-218c2747cc54 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:02:43.725909 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-435bb2dc-0a35-4821-8129-218c2747cc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:02:43.733563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 602/1203] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 06:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:02:43.884416 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a4f22-e347-4d1a-a8ba-3d3a3f9aa166 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:02:43.899526 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a4f22-e347-4d1a-a8ba-3d3a3f9aa166 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:02:43.904932 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01dbd6-a737-43ee-a5eb-1364c495764c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:02:43.905523 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01dbd6-a737-43ee-a5eb-1364c495764c service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:43.905800 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01dbd6-a737-43ee-a5eb-1364c495764c service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:43.906070 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01dbd6-a737-43ee-a5eb-1364c495764c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:43.914008 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c01dbd6-a737-43ee-a5eb-1364c495764c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:02:43.915563 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c01dbd6-a737-43ee-a5eb-1364c495764c service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:02:43.924672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 602/1204] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:02:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:02:47.858205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e234e82-eb07-4dde-a41e-3508bfbf914b None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:47.858779 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e234e82-eb07-4dde-a41e-3508bfbf914b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:47.858779 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e234e82-eb07-4dde-a41e-3508bfbf914b None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:47.860068 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 603/1205] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 06:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 06:02:47.865727 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d3457-6f79-4988-9a89-8941dba1e169 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:47.866170 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d3457-6f79-4988-9a89-8941dba1e169 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:47.866316 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d3457-6f79-4988-9a89-8941dba1e169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:47.886295 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-419d3457-6f79-4988-9a89-8941dba1e169 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:02:47.887488 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-419d3457-6f79-4988-9a89-8941dba1e169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:02:47.895314 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 603/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 06:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:02:48.180790 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e234e82-eb07-4dde-a41e-3508bfbf914b None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:02:48.182601 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1e234e82-eb07-4dde-a41e-3508bfbf914b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:02:48.261315 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e234e82-eb07-4dde-a41e-3508bfbf914b None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:02:48.264877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5945ab9-263d-41ee-898e-d039e0da906c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:02:48.265475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5945ab9-263d-41ee-898e-d039e0da906c service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:48.265751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5945ab9-263d-41ee-898e-d039e0da906c service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:48.266021 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5945ab9-263d-41ee-898e-d039e0da906c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:48.275248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5945ab9-263d-41ee-898e-d039e0da906c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:02:48.276719 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5945ab9-263d-41ee-898e-d039e0da906c service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:02:48.286024 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 604/1207] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:02:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:02:50.566125 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327850b9-bcdc-4b90-8b76-0533fc5953e6 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:50.566688 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327850b9-bcdc-4b90-8b76-0533fc5953e6 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:50.566688 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327850b9-bcdc-4b90-8b76-0533fc5953e6 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:50.568061 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 604/1208] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 06:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 06:02:50.575118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2383c91-f2f5-4d55-96a4-f4fe16c0df2d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:50.575555 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2383c91-f2f5-4d55-96a4-f4fe16c0df2d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:50.575978 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2383c91-f2f5-4d55-96a4-f4fe16c0df2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:50.597688 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-f2383c91-f2f5-4d55-96a4-f4fe16c0df2d None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:02:50.602455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f2383c91-f2f5-4d55-96a4-f4fe16c0df2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:02:50.612094 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 605/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 06:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:02:50.940898 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327850b9-bcdc-4b90-8b76-0533fc5953e6 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:02:50.946583 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=75 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:02:50.952989 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327850b9-bcdc-4b90-8b76-0533fc5953e6 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:02:50.956535 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f873fd-7eca-4717-b690-76409d5b62b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:02:50.957181 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f873fd-7eca-4717-b690-76409d5b62b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:50.957461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f873fd-7eca-4717-b690-76409d5b62b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:50.957733 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f873fd-7eca-4717-b690-76409d5b62b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:50.967333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f873fd-7eca-4717-b690-76409d5b62b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:02:50.968786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f873fd-7eca-4717-b690-76409d5b62b3 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:02:50.978908 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 605/1210] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:02:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:02:54.611896 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9e3c5-4bd8-4bca-954d-49e392ce7c2c None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:54.612430 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9e3c5-4bd8-4bca-954d-49e392ce7c2c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:54.612430 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9e3c5-4bd8-4bca-954d-49e392ce7c2c None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:54.613701 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 606/1211] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 06:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 06:02:54.621974 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ff7d2-2979-448f-9dcb-0d86fc0f97e0 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:54.622421 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ff7d2-2979-448f-9dcb-0d86fc0f97e0 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:54.622553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ff7d2-2979-448f-9dcb-0d86fc0f97e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:54.642425 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-df5ff7d2-2979-448f-9dcb-0d86fc0f97e0 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:02:54.643606 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-df5ff7d2-2979-448f-9dcb-0d86fc0f97e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:02:54.651432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 606/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 06:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:02:54.956768 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9e3c5-4bd8-4bca-954d-49e392ce7c2c None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:02:54.959023 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-42b9e3c5-4bd8-4bca-954d-49e392ce7c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:02:55.023413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:02:55.036587 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9e3c5-4bd8-4bca-954d-49e392ce7c2c None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:02:55.040166 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d06a88-8e95-4442-849c-35d8383ffd50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:02:55.040801 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d06a88-8e95-4442-849c-35d8383ffd50 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:55.041107 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d06a88-8e95-4442-849c-35d8383ffd50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:55.041386 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d06a88-8e95-4442-849c-35d8383ffd50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:55.050321 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d06a88-8e95-4442-849c-35d8383ffd50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:02:55.051820 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d06a88-8e95-4442-849c-35d8383ffd50 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:02:55.061267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 607/1213] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:02:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:02:57.268935 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a58b3d-3342-4d31-be5c-7c2ff7853274 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:57.269481 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a58b3d-3342-4d31-be5c-7c2ff7853274 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:57.269481 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a58b3d-3342-4d31-be5c-7c2ff7853274 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:57.271386 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 607/1214] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 06:02:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 06:02:57.276764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d406f8-2333-4343-a9cf-e1efa6908ea1 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:57.277231 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d406f8-2333-4343-a9cf-e1efa6908ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:57.277344 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d406f8-2333-4343-a9cf-e1efa6908ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:57.297586 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-59d406f8-2333-4343-a9cf-e1efa6908ea1 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:02:57.298789 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-59d406f8-2333-4343-a9cf-e1efa6908ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:02:57.306819 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 608/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 06:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:02:57.605063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a58b3d-3342-4d31-be5c-7c2ff7853274 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:02:57.606768 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f3a58b3d-3342-4d31-be5c-7c2ff7853274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:02:57.620268 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a58b3d-3342-4d31-be5c-7c2ff7853274 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:02:57.623785 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb15e2-cea1-4d05-8ae4-a1d0dcb0fe86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:02:57.624369 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb15e2-cea1-4d05-8ae4-a1d0dcb0fe86 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:02:57.624643 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb15e2-cea1-4d05-8ae4-a1d0dcb0fe86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:02:57.624937 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb15e2-cea1-4d05-8ae4-a1d0dcb0fe86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:02:57.634483 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb15e2-cea1-4d05-8ae4-a1d0dcb0fe86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:02:57.635889 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb15e2-cea1-4d05-8ae4-a1d0dcb0fe86 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:02:57.645142 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 608/1216] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:02:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:03:07.292305 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=11 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:03:07.640957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=14 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:22.215189 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c79c63-bf9d-45d8-b24d-f752b4ab0421 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:22.215825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c79c63-bf9d-45d8-b24d-f752b4ab0421 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:22.215825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c79c63-bf9d-45d8-b24d-f752b4ab0421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:22.240158 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-65c79c63-bf9d-45d8-b24d-f752b4ab0421 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:22.241534 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-65c79c63-bf9d-45d8-b24d-f752b4ab0421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:22.252549 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 609/1217] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:22.265055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db245fd-6fd7-4597-bf50-7bbc450d4452 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:22.265665 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db245fd-6fd7-4597-bf50-7bbc450d4452 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:22.265665 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db245fd-6fd7-4597-bf50-7bbc450d4452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:22.307224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-1db245fd-6fd7-4597-bf50-7bbc450d4452 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:22.308498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1db245fd-6fd7-4597-bf50-7bbc450d4452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:22.322070 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 609/1218] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:22.326487 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c79c63-bf9d-45d8-b24d-f752b4ab0421 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:22.345438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c79c63-bf9d-45d8-b24d-f752b4ab0421 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:22.361385 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e4a5e-32f5-4514-a904-c847bd85c364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:22.362013 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e4a5e-32f5-4514-a904-c847bd85c364 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:22.362291 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e4a5e-32f5-4514-a904-c847bd85c364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:22.362560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e4a5e-32f5-4514-a904-c847bd85c364 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:22.363206 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e4a5e-32f5-4514-a904-c847bd85c364 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:04:22.363506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e4a5e-32f5-4514-a904-c847bd85c364 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:22.368015 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e4a5e-32f5-4514-a904-c847bd85c364 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:22.381471 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 610/1219] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:04:22.390104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db245fd-6fd7-4597-bf50-7bbc450d4452 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:22.405227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db245fd-6fd7-4597-bf50-7bbc450d4452 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:22.415961 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42d42f-081e-4693-bae6-c238ff5843f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:22.416790 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42d42f-081e-4693-bae6-c238ff5843f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:22.417075 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42d42f-081e-4693-bae6-c238ff5843f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:22.417349 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42d42f-081e-4693-bae6-c238ff5843f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:22.418009 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42d42f-081e-4693-bae6-c238ff5843f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:22.419949 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42d42f-081e-4693-bae6-c238ff5843f0 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:22.499306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 610/1220] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:04:22 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:22.503608 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e4a5e-32f5-4514-a904-c847bd85c364 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:22.517103 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e4a5e-32f5-4514-a904-c847bd85c364 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:22.597558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b433ca6-d5b2-4356-8aca-064b0b4292f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:22.598181 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b433ca6-d5b2-4356-8aca-064b0b4292f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:22.598454 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b433ca6-d5b2-4356-8aca-064b0b4292f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:22.598722 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b433ca6-d5b2-4356-8aca-064b0b4292f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:22.599414 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b433ca6-d5b2-4356-8aca-064b0b4292f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:22.601426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b433ca6-d5b2-4356-8aca-064b0b4292f8 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:22.634958 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5139c798-ad15-43fc-a320-07363fb5ff7a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:22.635311 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5139c798-ad15-43fc-a320-07363fb5ff7a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:22.635528 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5139c798-ad15-43fc-a320-07363fb5ff7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:22.638351 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 611/1221] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:04:22 2022] POST /identity/v3/users => generated 292 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:22.642303 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b433ca6-d5b2-4356-8aca-064b0b4292f8 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:22.653389 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b433ca6-d5b2-4356-8aca-064b0b4292f8 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:22.661891 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d0198-3db4-4ed6-bfa9-1273b47e1613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:22.662600 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d0198-3db4-4ed6-bfa9-1273b47e1613 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:22.662899 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d0198-3db4-4ed6-bfa9-1273b47e1613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:22.663136 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d0198-3db4-4ed6-bfa9-1273b47e1613 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:22.663777 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d0198-3db4-4ed6-bfa9-1273b47e1613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:22.665507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d0198-3db4-4ed6-bfa9-1273b47e1613 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:22.671514 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-5139c798-ad15-43fc-a320-07363fb5ff7a None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:22.672856 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5139c798-ad15-43fc-a320-07363fb5ff7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:22.681794 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 612/1222] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:22.687761 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d0198-3db4-4ed6-bfa9-1273b47e1613 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:22.687943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 611/1223] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:22.695595 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d12d9b-e10b-481b-9209-adfbde2a6cde None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:22.695901 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d12d9b-e10b-481b-9209-adfbde2a6cde None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:22.696153 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d12d9b-e10b-481b-9209-adfbde2a6cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:22.702619 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d0198-3db4-4ed6-bfa9-1273b47e1613 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:22.721395 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03a929-855b-488b-bf0e-57f31c218ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:22.722018 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03a929-855b-488b-bf0e-57f31c218ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:22.722309 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03a929-855b-488b-bf0e-57f31c218ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:22.722562 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03a929-855b-488b-bf0e-57f31c218ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:22.723206 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd03a929-855b-488b-bf0e-57f31c218ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:22.724681 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-08d12d9b-e10b-481b-9209-adfbde2a6cde None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:22.725996 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd03a929-855b-488b-bf0e-57f31c218ca1 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:22.726148 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-08d12d9b-e10b-481b-9209-adfbde2a6cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:22.737242 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 613/1224] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:22.738838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b3d0d-77aa-4f1b-ad80-1e60f56cf809 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:22.739123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b3d0d-77aa-4f1b-ad80-1e60f56cf809 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:22.739394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b3d0d-77aa-4f1b-ad80-1e60f56cf809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:22.742367 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 612/1225] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:04:22.745238 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d12d9b-e10b-481b-9209-adfbde2a6cde None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:22.756792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d12d9b-e10b-481b-9209-adfbde2a6cde None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:22.766335 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7c412-8e47-4321-a23c-01b352e8dba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:22.767093 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7c412-8e47-4321-a23c-01b352e8dba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:22.767383 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7c412-8e47-4321-a23c-01b352e8dba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:22.767661 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7c412-8e47-4321-a23c-01b352e8dba7 admin admin] PATH_INFO: `/v3/projects/b90087118fe040389f291d846461c0a7/users/2c6be23065c643939055a3de375187d5/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:22.786630 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-8f3b3d0d-77aa-4f1b-ad80-1e60f56cf809 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:22.787828 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc7c412-8e47-4321-a23c-01b352e8dba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b90087118fe040389f291d846461c0a7, user_id=2c6be23065c643939055a3de375187d5, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:22.788160 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8f3b3d0d-77aa-4f1b-ad80-1e60f56cf809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:22.791019 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc7c412-8e47-4321-a23c-01b352e8dba7 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:22.797051 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 614/1226] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:22.797917 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b3d0d-77aa-4f1b-ad80-1e60f56cf809 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:22.809160 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b3d0d-77aa-4f1b-ad80-1e60f56cf809 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:22.822578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 613/1227] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:04:22 2022] PUT /identity/v3/projects/b90087118fe040389f291d846461c0a7/users/2c6be23065c643939055a3de375187d5/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:04:22.828967 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac3715-3aa7-43b9-b638-ac91d14c64aa None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:22.829265 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac3715-3aa7-43b9-b638-ac91d14c64aa None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:22.830469 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac3715-3aa7-43b9-b638-ac91d14c64aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:22.861287 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f5ac3715-3aa7-43b9-b638-ac91d14c64aa None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:22.862718 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f5ac3715-3aa7-43b9-b638-ac91d14c64aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:22.888717 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b8224-2b77-4cb2-b43a-6eb99fd07819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:22.889340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b8224-2b77-4cb2-b43a-6eb99fd07819 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:22.889649 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b8224-2b77-4cb2-b43a-6eb99fd07819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:22.889948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b8224-2b77-4cb2-b43a-6eb99fd07819 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:22.890590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b8224-2b77-4cb2-b43a-6eb99fd07819 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:04:22.890892 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b8224-2b77-4cb2-b43a-6eb99fd07819 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:22.892470 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b8224-2b77-4cb2-b43a-6eb99fd07819 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:22.912397 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 615/1228] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:04:22.912397 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51939850-d4df-4c17-b152-abd71a1d55a3 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:22.912397 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51939850-d4df-4c17-b152-abd71a1d55a3 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:22.912397 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51939850-d4df-4c17-b152-abd71a1d55a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:22.931203 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-51939850-d4df-4c17-b152-abd71a1d55a3 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:22.932409 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-51939850-d4df-4c17-b152-abd71a1d55a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:22.937219 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 614/1229] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:22.938906 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae7d41-e3d2-48e2-905e-6d7f7af41439 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:22.939488 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae7d41-e3d2-48e2-905e-6d7f7af41439 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:22.939982 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae7d41-e3d2-48e2-905e-6d7f7af41439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:22.940448 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 616/1230] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:22.941802 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76803d6a-3ce8-4dd6-bfe2-0ddb2dae22b2 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:22.942083 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76803d6a-3ce8-4dd6-bfe2-0ddb2dae22b2 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:22.942353 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76803d6a-3ce8-4dd6-bfe2-0ddb2dae22b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:22.964920 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-7cae7d41-e3d2-48e2-905e-6d7f7af41439 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:22.965758 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-76803d6a-3ce8-4dd6-bfe2-0ddb2dae22b2 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:22.967313 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7cae7d41-e3d2-48e2-905e-6d7f7af41439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:22.967503 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-76803d6a-3ce8-4dd6-bfe2-0ddb2dae22b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:22.976281 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 617/1231] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:22.977638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 615/1232] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:22.979171 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e3839-4e70-4fff-b36a-bc8a0cb8284f None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:22.979664 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e3839-4e70-4fff-b36a-bc8a0cb8284f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:22.980157 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e3839-4e70-4fff-b36a-bc8a0cb8284f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:22.980991 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9cd26-c7c2-4cc1-92ff-b10699954b01 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:22.981297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9cd26-c7c2-4cc1-92ff-b10699954b01 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:22.981632 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9cd26-c7c2-4cc1-92ff-b10699954b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.002751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-c7a9cd26-c7c2-4cc1-92ff-b10699954b01 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:23.004402 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c7a9cd26-c7c2-4cc1-92ff-b10699954b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:23.008816 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-8b8e3839-4e70-4fff-b36a-bc8a0cb8284f None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:23.010182 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8b8e3839-4e70-4fff-b36a-bc8a0cb8284f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:23.016336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 618/1233] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:23.017185 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9cd26-c7c2-4cc1-92ff-b10699954b01 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:23.023640 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 616/1234] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:23.025781 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525e218-d0a5-4674-96f4-b0e4455caf45 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.026216 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525e218-d0a5-4674-96f4-b0e4455caf45 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.026351 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525e218-d0a5-4674-96f4-b0e4455caf45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.030142 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9cd26-c7c2-4cc1-92ff-b10699954b01 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:23.036529 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715585bb-63db-406e-9d4b-f75e7a991a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:23.037544 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715585bb-63db-406e-9d4b-f75e7a991a24 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.037670 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715585bb-63db-406e-9d4b-f75e7a991a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.037919 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715585bb-63db-406e-9d4b-f75e7a991a24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.038644 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715585bb-63db-406e-9d4b-f75e7a991a24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:23.039785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: WARNING py.warnings [None req-715585bb-63db-406e-9d4b-f75e7a991a24 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 06:04:23.039785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: warnings.warn(msg) Oct 20 06:04:23.039785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: Oct 20 06:04:23.041130 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715585bb-63db-406e-9d4b-f75e7a991a24 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:23.052117 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-e525e218-d0a5-4674-96f4-b0e4455caf45 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:23.054084 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e525e218-d0a5-4674-96f4-b0e4455caf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:23.057866 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:23.063000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 617/1235] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:23.064460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64f23a-705c-4a4e-8826-058270342093 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.064828 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64f23a-705c-4a4e-8826-058270342093 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.065178 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64f23a-705c-4a4e-8826-058270342093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.074510 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 619/1236] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:04:23 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:23.075497 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715585bb-63db-406e-9d4b-f75e7a991a24 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:23.086022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-6f64f23a-705c-4a4e-8826-058270342093 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:23.087239 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6f64f23a-705c-4a4e-8826-058270342093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:23.142342 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715585bb-63db-406e-9d4b-f75e7a991a24 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:23.146134 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 618/1237] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:23.147278 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c400724-a3c1-4076-9431-8fe3138a0c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:23.147424 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95a9b0-375c-465a-99c5-ecae22b0a801 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.147642 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95a9b0-375c-465a-99c5-ecae22b0a801 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.147912 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95a9b0-375c-465a-99c5-ecae22b0a801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.148499 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c400724-a3c1-4076-9431-8fe3138a0c25 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.148814 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c400724-a3c1-4076-9431-8fe3138a0c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.149088 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c400724-a3c1-4076-9431-8fe3138a0c25 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.149917 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c400724-a3c1-4076-9431-8fe3138a0c25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:04:23.150190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c400724-a3c1-4076-9431-8fe3138a0c25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:23.151780 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c400724-a3c1-4076-9431-8fe3138a0c25 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:23.160785 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:23.172376 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 620/1238] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:04:23.172376 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c400724-a3c1-4076-9431-8fe3138a0c25 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:23.172670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-cc95a9b0-375c-465a-99c5-ecae22b0a801 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:23.172670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-cc95a9b0-375c-465a-99c5-ecae22b0a801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:23.179543 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 619/1239] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:23.180828 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8819ba9-16ab-4fa6-8e6a-aa7c10abf2d7 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.181106 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8819ba9-16ab-4fa6-8e6a-aa7c10abf2d7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.181381 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8819ba9-16ab-4fa6-8e6a-aa7c10abf2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.200329 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-b8819ba9-16ab-4fa6-8e6a-aa7c10abf2d7 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:23.201547 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b8819ba9-16ab-4fa6-8e6a-aa7c10abf2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:23.210563 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 620/1240] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:23.214300 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9a9a8-ae7e-4495-bf31-03c63a908374 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.214300 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9a9a8-ae7e-4495-bf31-03c63a908374 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.214300 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9a9a8-ae7e-4495-bf31-03c63a908374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.225982 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c400724-a3c1-4076-9431-8fe3138a0c25 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:23.230026 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c689b-b229-45ff-b60a-7d5d171371dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:23.230629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c689b-b229-45ff-b60a-7d5d171371dd service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.230903 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c689b-b229-45ff-b60a-7d5d171371dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.231172 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c689b-b229-45ff-b60a-7d5d171371dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.232297 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3ae9a9a8-ae7e-4495-bf31-03c63a908374 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:23.233495 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3ae9a9a8-ae7e-4495-bf31-03c63a908374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:23.240930 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c689b-b229-45ff-b60a-7d5d171371dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:23.241349 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 621/1241] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:23.242388 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c689b-b229-45ff-b60a-7d5d171371dd service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:23.245620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9a9a8-ae7e-4495-bf31-03c63a908374 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:23.252093 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 621/1242] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:04:23.252879 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c689b-b229-45ff-b60a-7d5d171371dd service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:23.254712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9a9a8-ae7e-4495-bf31-03c63a908374 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:23.260159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55b2e0-5307-4ffe-9848-a06518ad86fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:23.260776 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55b2e0-5307-4ffe-9848-a06518ad86fa admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.261055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55b2e0-5307-4ffe-9848-a06518ad86fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.261327 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55b2e0-5307-4ffe-9848-a06518ad86fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.261958 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55b2e0-5307-4ffe-9848-a06518ad86fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:04:23.262253 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55b2e0-5307-4ffe-9848-a06518ad86fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:23.263260 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: WARNING py.warnings [None req-da55b2e0-5307-4ffe-9848-a06518ad86fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 20 06:04:23.263260 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: warnings.warn(msg) Oct 20 06:04:23.263260 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: Oct 20 06:04:23.263480 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c689b-b229-45ff-b60a-7d5d171371dd service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:23.264124 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55b2e0-5307-4ffe-9848-a06518ad86fa admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:23.268132 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f542b5-570f-4aa2-a167-3b9584a0f51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:23.268746 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f542b5-570f-4aa2-a167-3b9584a0f51c admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.269026 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f542b5-570f-4aa2-a167-3b9584a0f51c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.269295 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f542b5-570f-4aa2-a167-3b9584a0f51c admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.270344 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f542b5-570f-4aa2-a167-3b9584a0f51c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:23.273008 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f542b5-570f-4aa2-a167-3b9584a0f51c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:23.284373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 622/1243] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:04:23.284373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55b2e0-5307-4ffe-9848-a06518ad86fa admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:23.295139 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55b2e0-5307-4ffe-9848-a06518ad86fa admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:23.298804 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 622/1244] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:04:23 2022] POST /identity/v3/users => generated 283 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:23.303647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f542b5-570f-4aa2-a167-3b9584a0f51c admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:23.305853 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dcdb5e-83e7-4f66-a6c4-aa87f8b141f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:23.306548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dcdb5e-83e7-4f66-a6c4-aa87f8b141f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.307138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dcdb5e-83e7-4f66-a6c4-aa87f8b141f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.307472 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dcdb5e-83e7-4f66-a6c4-aa87f8b141f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.308191 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dcdb5e-83e7-4f66-a6c4-aa87f8b141f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:04:23.308594 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dcdb5e-83e7-4f66-a6c4-aa87f8b141f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:23.310341 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dcdb5e-83e7-4f66-a6c4-aa87f8b141f9 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:23.314214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f542b5-570f-4aa2-a167-3b9584a0f51c admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:23.319889 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e74016-03b1-46a8-80d5-4065a8e1f569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:23.320538 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e74016-03b1-46a8-80d5-4065a8e1f569 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.321658 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e74016-03b1-46a8-80d5-4065a8e1f569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.321658 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e74016-03b1-46a8-80d5-4065a8e1f569 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.321840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e74016-03b1-46a8-80d5-4065a8e1f569 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:23.323647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e74016-03b1-46a8-80d5-4065a8e1f569 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:23.331914 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 623/1245] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:04:23.331914 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033a51b-b0bb-4069-92de-6a8a2a764d0c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.331914 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033a51b-b0bb-4069-92de-6a8a2a764d0c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.331914 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033a51b-b0bb-4069-92de-6a8a2a764d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.350675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f033a51b-b0bb-4069-92de-6a8a2a764d0c None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:23.352180 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f033a51b-b0bb-4069-92de-6a8a2a764d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:23.360426 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 624/1246] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:23.361436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033a51b-b0bb-4069-92de-6a8a2a764d0c None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:23.362390 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 623/1247] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:04:23 2022] POST /identity/v3/projects => generated 391 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:23.363297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e74016-03b1-46a8-80d5-4065a8e1f569 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:23.374776 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033a51b-b0bb-4069-92de-6a8a2a764d0c None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:23.431752 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:23.447083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f851fff-e130-494c-9172-35e2c70c1660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:23.450002 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f851fff-e130-494c-9172-35e2c70c1660 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.450002 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f851fff-e130-494c-9172-35e2c70c1660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.450002 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f851fff-e130-494c-9172-35e2c70c1660 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.450002 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f851fff-e130-494c-9172-35e2c70c1660 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:04:23.450457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f851fff-e130-494c-9172-35e2c70c1660 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:23.450660 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e74016-03b1-46a8-80d5-4065a8e1f569 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:23.452191 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f851fff-e130-494c-9172-35e2c70c1660 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:23.460309 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec79b1e-677b-45a0-a756-4669ea169394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:23.460963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec79b1e-677b-45a0-a756-4669ea169394 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.461244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec79b1e-677b-45a0-a756-4669ea169394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.461542 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec79b1e-677b-45a0-a756-4669ea169394 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.462154 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec79b1e-677b-45a0-a756-4669ea169394 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:04:23.462456 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec79b1e-677b-45a0-a756-4669ea169394 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:23.464140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec79b1e-677b-45a0-a756-4669ea169394 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:23.477072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 625/1248] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:04:23.484219 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 624/1249] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:04:23.484219 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec79b1e-677b-45a0-a756-4669ea169394 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:23.485549 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f851fff-e130-494c-9172-35e2c70c1660 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:23.493307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec79b1e-677b-45a0-a756-4669ea169394 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:23.496447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f851fff-e130-494c-9172-35e2c70c1660 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:23.498618 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f75bfb-5103-42c6-a749-3db903eef8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:23.499227 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f75bfb-5103-42c6-a749-3db903eef8c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.499848 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f75bfb-5103-42c6-a749-3db903eef8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.500118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f75bfb-5103-42c6-a749-3db903eef8c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.500776 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f75bfb-5103-42c6-a749-3db903eef8c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:23.502772 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f75bfb-5103-42c6-a749-3db903eef8c6 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:23.503523 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf901937-f0f6-477f-b0fc-7f3b2846976d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:23.504143 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf901937-f0f6-477f-b0fc-7f3b2846976d admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.504522 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf901937-f0f6-477f-b0fc-7f3b2846976d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.506147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf901937-f0f6-477f-b0fc-7f3b2846976d admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.506910 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf901937-f0f6-477f-b0fc-7f3b2846976d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:23.509919 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf901937-f0f6-477f-b0fc-7f3b2846976d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:23.526854 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 626/1250] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:23.527762 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf901937-f0f6-477f-b0fc-7f3b2846976d admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:23.534990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 625/1251] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:04:23 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:23.535944 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f75bfb-5103-42c6-a749-3db903eef8c6 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:23.538251 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf901937-f0f6-477f-b0fc-7f3b2846976d admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:23.613442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee777bd-1147-47bd-a14b-815ff59e3073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:23.614260 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee777bd-1147-47bd-a14b-815ff59e3073 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.617243 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee777bd-1147-47bd-a14b-815ff59e3073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.617598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee777bd-1147-47bd-a14b-815ff59e3073 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.618479 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee777bd-1147-47bd-a14b-815ff59e3073 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:23.620089 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f75bfb-5103-42c6-a749-3db903eef8c6 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:23.620849 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee777bd-1147-47bd-a14b-815ff59e3073 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:23.635584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e269b1-c9f1-4240-acb8-7582f6013a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:23.636246 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e269b1-c9f1-4240-acb8-7582f6013a98 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.636586 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e269b1-c9f1-4240-acb8-7582f6013a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.636908 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e269b1-c9f1-4240-acb8-7582f6013a98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.637630 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e269b1-c9f1-4240-acb8-7582f6013a98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:04:23.638276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e269b1-c9f1-4240-acb8-7582f6013a98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:23.640466 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e269b1-c9f1-4240-acb8-7582f6013a98 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:23.658569 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 627/1252] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:04:23 2022] POST /identity/v3/projects => generated 389 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:23.658796 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 626/1253] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:04:23.658916 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee777bd-1147-47bd-a14b-815ff59e3073 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:23.659032 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e269b1-c9f1-4240-acb8-7582f6013a98 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:23.678894 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e269b1-c9f1-4240-acb8-7582f6013a98 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:23.689185 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:23.741576 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee777bd-1147-47bd-a14b-815ff59e3073 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:23.746529 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ec449-cfe3-4b70-af6a-6be3262eb62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:23.747147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ec449-cfe3-4b70-af6a-6be3262eb62f admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.747424 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ec449-cfe3-4b70-af6a-6be3262eb62f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.747692 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ec449-cfe3-4b70-af6a-6be3262eb62f admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.748318 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ec449-cfe3-4b70-af6a-6be3262eb62f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:23.749521 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: WARNING py.warnings [None req-b67ec449-cfe3-4b70-af6a-6be3262eb62f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 06:04:23.749521 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: warnings.warn(msg) Oct 20 06:04:23.749521 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: Oct 20 06:04:23.750751 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ec449-cfe3-4b70-af6a-6be3262eb62f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:23.767544 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd3a21-f7ff-454f-b868-051f21d49f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:23.768654 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd3a21-f7ff-454f-b868-051f21d49f2b admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.769185 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd3a21-f7ff-454f-b868-051f21d49f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.769971 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd3a21-f7ff-454f-b868-051f21d49f2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.770878 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd3a21-f7ff-454f-b868-051f21d49f2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:23.774012 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd3a21-f7ff-454f-b868-051f21d49f2b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:23.775955 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 628/1254] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:04:23 2022] POST /identity/v3/users => generated 296 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:23.776879 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ec449-cfe3-4b70-af6a-6be3262eb62f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:23.789676 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ec449-cfe3-4b70-af6a-6be3262eb62f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:23.795094 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce521297-30c1-4bc6-acc6-2f9eee0e96c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:23.795874 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce521297-30c1-4bc6-acc6-2f9eee0e96c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.796208 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce521297-30c1-4bc6-acc6-2f9eee0e96c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.796500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce521297-30c1-4bc6-acc6-2f9eee0e96c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.797399 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce521297-30c1-4bc6-acc6-2f9eee0e96c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:23.799953 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce521297-30c1-4bc6-acc6-2f9eee0e96c7 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:23.807966 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 627/1255] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:04:23 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:23.808800 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd3a21-f7ff-454f-b868-051f21d49f2b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:23.836823 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 629/1256] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:04:23 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:23.836823 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce521297-30c1-4bc6-acc6-2f9eee0e96c7 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:23.859402 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:23.880269 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd3a21-f7ff-454f-b868-051f21d49f2b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:23.906541 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce521297-30c1-4bc6-acc6-2f9eee0e96c7 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:23.911943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3146dd-3197-4754-aec8-bb86feb8bc61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:23.912643 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3146dd-3197-4754-aec8-bb86feb8bc61 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.913216 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3146dd-3197-4754-aec8-bb86feb8bc61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.913560 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3146dd-3197-4754-aec8-bb86feb8bc61 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.914273 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3146dd-3197-4754-aec8-bb86feb8bc61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:23.916288 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3146dd-3197-4754-aec8-bb86feb8bc61 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:23.945727 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 630/1257] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:04:23 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:23.946471 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3146dd-3197-4754-aec8-bb86feb8bc61 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:23.953841 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6867f4-f1a8-4db3-878c-6be11a7da4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:23.954470 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6867f4-f1a8-4db3-878c-6be11a7da4ac admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.954753 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6867f4-f1a8-4db3-878c-6be11a7da4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.955122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6867f4-f1a8-4db3-878c-6be11a7da4ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.955666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6867f4-f1a8-4db3-878c-6be11a7da4ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:23.957355 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6867f4-f1a8-4db3-878c-6be11a7da4ac admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:23.957863 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3146dd-3197-4754-aec8-bb86feb8bc61 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:23.964095 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52340de5-4372-4d2a-8970-943e1e029ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:23.964733 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52340de5-4372-4d2a-8970-943e1e029ec2 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.965068 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52340de5-4372-4d2a-8970-943e1e029ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.965302 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52340de5-4372-4d2a-8970-943e1e029ec2 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.965970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52340de5-4372-4d2a-8970-943e1e029ec2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:23.967894 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52340de5-4372-4d2a-8970-943e1e029ec2 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:23.968560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 628/1258] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:23.969369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6867f4-f1a8-4db3-878c-6be11a7da4ac admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:23.980328 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6867f4-f1a8-4db3-878c-6be11a7da4ac admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:23.989594 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f821d1-7938-4e9a-8a62-ba3c3e6adea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:23.990605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f821d1-7938-4e9a-8a62-ba3c3e6adea0 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:23.991039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f821d1-7938-4e9a-8a62-ba3c3e6adea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:23.991407 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f821d1-7938-4e9a-8a62-ba3c3e6adea0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:23.992292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f821d1-7938-4e9a-8a62-ba3c3e6adea0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:23.994502 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f821d1-7938-4e9a-8a62-ba3c3e6adea0 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:24.004458 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 631/1259] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:04:23 2022] POST /identity/v3/users => generated 295 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:24.007185 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52340de5-4372-4d2a-8970-943e1e029ec2 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:24.018838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52340de5-4372-4d2a-8970-943e1e029ec2 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:24.026541 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac3085-8192-450c-a56d-c52bce63e500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:24.027293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac3085-8192-450c-a56d-c52bce63e500 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:24.027638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac3085-8192-450c-a56d-c52bce63e500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:24.027977 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac3085-8192-450c-a56d-c52bce63e500 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:24.028827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac3085-8192-450c-a56d-c52bce63e500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:24.030702 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac3085-8192-450c-a56d-c52bce63e500 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:24.035179 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 629/1260] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:04:23 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:24.036079 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f821d1-7938-4e9a-8a62-ba3c3e6adea0 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:24.054818 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 632/1261] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:24.057099 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac3085-8192-450c-a56d-c52bce63e500 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:24.071527 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac3085-8192-450c-a56d-c52bce63e500 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:24.108377 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f821d1-7938-4e9a-8a62-ba3c3e6adea0 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:24.113300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7826fa6-7b3d-43dc-9668-148f781dafdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:24.113916 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7826fa6-7b3d-43dc-9668-148f781dafdd admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:24.114197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7826fa6-7b3d-43dc-9668-148f781dafdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:24.114473 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7826fa6-7b3d-43dc-9668-148f781dafdd admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:24.115260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7826fa6-7b3d-43dc-9668-148f781dafdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:24.117390 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7826fa6-7b3d-43dc-9668-148f781dafdd admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:24.140351 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:24.155817 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 630/1262] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:04:24 2022] POST /identity/v3/users => generated 287 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:24.156832 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7826fa6-7b3d-43dc-9668-148f781dafdd admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:24.157504 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a69099-c0bb-49a9-b845-4945f292667c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:24.158257 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a69099-c0bb-49a9-b845-4945f292667c admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:24.158544 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a69099-c0bb-49a9-b845-4945f292667c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:24.158970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a69099-c0bb-49a9-b845-4945f292667c admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:24.159618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a69099-c0bb-49a9-b845-4945f292667c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:24.162466 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a69099-c0bb-49a9-b845-4945f292667c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:24.168953 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7826fa6-7b3d-43dc-9668-148f781dafdd admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:24.174053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e41e6-b1d9-446f-92ac-a996f701078b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:24.174672 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e41e6-b1d9-446f-92ac-a996f701078b admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:24.174974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e41e6-b1d9-446f-92ac-a996f701078b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:24.175332 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e41e6-b1d9-446f-92ac-a996f701078b admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:24.176638 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e41e6-b1d9-446f-92ac-a996f701078b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:24.179239 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e41e6-b1d9-446f-92ac-a996f701078b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:24.192170 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 633/1263] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:04:24 2022] POST /identity/v3/users => generated 289 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:24.192384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 631/1264] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:24.192481 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a69099-c0bb-49a9-b845-4945f292667c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:24.195185 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e41e6-b1d9-446f-92ac-a996f701078b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:24.205219 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a69099-c0bb-49a9-b845-4945f292667c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:24.207245 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e41e6-b1d9-446f-92ac-a996f701078b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:24.212508 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ada4c7-8b8f-4feb-b847-41fff14e5ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:24.213044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ed44f-d7f8-42a7-b45f-51700634aaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:24.213247 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ada4c7-8b8f-4feb-b847-41fff14e5ac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:24.213651 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ed44f-d7f8-42a7-b45f-51700634aaf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:24.213762 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ada4c7-8b8f-4feb-b847-41fff14e5ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:24.213921 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ed44f-d7f8-42a7-b45f-51700634aaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:24.214165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ada4c7-8b8f-4feb-b847-41fff14e5ac7 admin admin] PATH_INFO: `/v3/projects/7f038a048ecc436d857d0cb2866f5c59/users/645232f827e544219e703c1a75551c89/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:24.214284 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ed44f-d7f8-42a7-b45f-51700634aaf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:24.214825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ed44f-d7f8-42a7-b45f-51700634aaf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:24.216446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ed44f-d7f8-42a7-b45f-51700634aaf6 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:24.233177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 632/1265] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:24.233809 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ed44f-d7f8-42a7-b45f-51700634aaf6 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:24.241365 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ada4c7-8b8f-4feb-b847-41fff14e5ac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f038a048ecc436d857d0cb2866f5c59, user_id=645232f827e544219e703c1a75551c89, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:24.244081 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ed44f-d7f8-42a7-b45f-51700634aaf6 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:24.244200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ada4c7-8b8f-4feb-b847-41fff14e5ac7 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:24.255103 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f49d0e-4d27-487c-8181-2023cd1ca40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:24.255922 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f49d0e-4d27-487c-8181-2023cd1ca40b admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:24.256200 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f49d0e-4d27-487c-8181-2023cd1ca40b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:24.256524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f49d0e-4d27-487c-8181-2023cd1ca40b admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:24.257200 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f49d0e-4d27-487c-8181-2023cd1ca40b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:24.261151 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f49d0e-4d27-487c-8181-2023cd1ca40b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:24.276545 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 634/1266] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:04:24 2022] PUT /identity/v3/projects/7f038a048ecc436d857d0cb2866f5c59/users/645232f827e544219e703c1a75551c89/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:04:24.276545 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ada4c7-8b8f-4feb-b847-41fff14e5ac7 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:24.298837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 633/1267] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:04:24 2022] POST /identity/v3/users => generated 288 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:24.302948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f49d0e-4d27-487c-8181-2023cd1ca40b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:24.313356 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f49d0e-4d27-487c-8181-2023cd1ca40b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:24.323071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:24.373929 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ada4c7-8b8f-4feb-b847-41fff14e5ac7 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:24.380329 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6affa57-f391-42ba-a6b1-dc1ea9889db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:24.382086 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6affa57-f391-42ba-a6b1-dc1ea9889db0 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:24.382517 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6affa57-f391-42ba-a6b1-dc1ea9889db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:24.382991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6affa57-f391-42ba-a6b1-dc1ea9889db0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:24.384861 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6affa57-f391-42ba-a6b1-dc1ea9889db0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:24.385225 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c4d8d-8ef5-4eb1-9767-98f555be3a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:24.385825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c4d8d-8ef5-4eb1-9767-98f555be3a6f admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:24.386099 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c4d8d-8ef5-4eb1-9767-98f555be3a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:24.386810 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c4d8d-8ef5-4eb1-9767-98f555be3a6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:24.386998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c4d8d-8ef5-4eb1-9767-98f555be3a6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:24.388609 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c4d8d-8ef5-4eb1-9767-98f555be3a6f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:24.389042 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6affa57-f391-42ba-a6b1-dc1ea9889db0 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:24.400516 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 634/1268] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:24.401398 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c4d8d-8ef5-4eb1-9767-98f555be3a6f admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:24.407133 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 635/1269] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:24.408943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6affa57-f391-42ba-a6b1-dc1ea9889db0 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:24.412267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c4d8d-8ef5-4eb1-9767-98f555be3a6f admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:24.418223 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cbd28-da43-4c9d-9c66-a9aae13cb0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:24.418835 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cbd28-da43-4c9d-9c66-a9aae13cb0f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:24.419106 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cbd28-da43-4c9d-9c66-a9aae13cb0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:24.419674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cbd28-da43-4c9d-9c66-a9aae13cb0f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:24.420000 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0cbd28-da43-4c9d-9c66-a9aae13cb0f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:24.421626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0cbd28-da43-4c9d-9c66-a9aae13cb0f8 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:24.427658 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6affa57-f391-42ba-a6b1-dc1ea9889db0 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:24.432137 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 635/1270] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:24.435477 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bd3fe-5d41-4166-97d3-52b8b8f788a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:24.435881 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cbd28-da43-4c9d-9c66-a9aae13cb0f8 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:24.436798 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bd3fe-5d41-4166-97d3-52b8b8f788a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:24.437075 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bd3fe-5d41-4166-97d3-52b8b8f788a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:24.437488 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bd3fe-5d41-4166-97d3-52b8b8f788a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:24.438230 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27bd3fe-5d41-4166-97d3-52b8b8f788a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:24.441170 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27bd3fe-5d41-4166-97d3-52b8b8f788a2 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:24.448260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cbd28-da43-4c9d-9c66-a9aae13cb0f8 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:24.454866 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50976158-7ad1-4248-8ffa-a7d8062d7cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:24.455479 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50976158-7ad1-4248-8ffa-a7d8062d7cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:24.455803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50976158-7ad1-4248-8ffa-a7d8062d7cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:24.456077 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50976158-7ad1-4248-8ffa-a7d8062d7cc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:24.456859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 636/1271] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:24.457000 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50976158-7ad1-4248-8ffa-a7d8062d7cc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:24.458121 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1585a41-6020-426e-ad65-6a12bbde58a6 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:24.458373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50976158-7ad1-4248-8ffa-a7d8062d7cc1 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:24.458531 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1585a41-6020-426e-ad65-6a12bbde58a6 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:24.459227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1585a41-6020-426e-ad65-6a12bbde58a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:24.473174 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 636/1272] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:24.473174 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50976158-7ad1-4248-8ffa-a7d8062d7cc1 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:24.487717 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50976158-7ad1-4248-8ffa-a7d8062d7cc1 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:24.493398 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d500f-6604-4196-99c3-58a00a7e0d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:24.493579 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-b1585a41-6020-426e-ad65-6a12bbde58a6 None None] MFA Rules not processed for user `645232f827e544219e703c1a75551c89`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:24.494066 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d500f-6604-4196-99c3-58a00a7e0d88 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:24.494345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d500f-6604-4196-99c3-58a00a7e0d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:24.494610 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d500f-6604-4196-99c3-58a00a7e0d88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:24.494730 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b1585a41-6020-426e-ad65-6a12bbde58a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:24.495293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d500f-6604-4196-99c3-58a00a7e0d88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:24.496954 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d500f-6604-4196-99c3-58a00a7e0d88 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:24.514805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 637/1273] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:24.515841 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d500f-6604-4196-99c3-58a00a7e0d88 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:24.526452 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d500f-6604-4196-99c3-58a00a7e0d88 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:24.532171 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a2122-2f60-48a3-a7af-268ba73005fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:24.532843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a2122-2f60-48a3-a7af-268ba73005fe admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:24.533117 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a2122-2f60-48a3-a7af-268ba73005fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:24.533384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a2122-2f60-48a3-a7af-268ba73005fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:24.534010 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a2122-2f60-48a3-a7af-268ba73005fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:24.536015 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a2122-2f60-48a3-a7af-268ba73005fe admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:24.549224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 638/1274] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:24.550332 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a2122-2f60-48a3-a7af-268ba73005fe admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:24.562332 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a2122-2f60-48a3-a7af-268ba73005fe admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:24.569623 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70482b-a9e5-4b18-9312-d74d963cc459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:24.570297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70482b-a9e5-4b18-9312-d74d963cc459 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:24.570571 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70482b-a9e5-4b18-9312-d74d963cc459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:24.570947 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70482b-a9e5-4b18-9312-d74d963cc459 admin admin] PATH_INFO: `/v3/projects/9f0d80988ca941acb8ad4318338dfbea/users/57d4cbd2a2154e5d925462df7c77815a/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:24.591690 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e70482b-a9e5-4b18-9312-d74d963cc459 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f0d80988ca941acb8ad4318338dfbea, user_id=57d4cbd2a2154e5d925462df7c77815a, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:24.591914 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 637/1275] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:24.593381 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1585a41-6020-426e-ad65-6a12bbde58a6 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:24.603318 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e70482b-a9e5-4b18-9312-d74d963cc459 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:24.615246 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1585a41-6020-426e-ad65-6a12bbde58a6 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:24.623531 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 639/1276] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:04:24 2022] PUT /identity/v3/projects/9f0d80988ca941acb8ad4318338dfbea/users/57d4cbd2a2154e5d925462df7c77815a/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:04:24.624448 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70482b-a9e5-4b18-9312-d74d963cc459 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:24.668989 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:24.716029 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2cf08-659b-41d2-964a-5f5288d89480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:24.717000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2cf08-659b-41d2-964a-5f5288d89480 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:24.717705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2cf08-659b-41d2-964a-5f5288d89480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:24.718134 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2cf08-659b-41d2-964a-5f5288d89480 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:24.719694 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2cf08-659b-41d2-964a-5f5288d89480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:24.722261 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2cf08-659b-41d2-964a-5f5288d89480 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:24.725762 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70482b-a9e5-4b18-9312-d74d963cc459 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:24.739831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 638/1277] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:24.739831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2cf08-659b-41d2-964a-5f5288d89480 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:24.742221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca7288-b644-4b2d-91c4-a68c01e2344d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:24.742973 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca7288-b644-4b2d-91c4-a68c01e2344d admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:24.743248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca7288-b644-4b2d-91c4-a68c01e2344d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:24.743518 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca7288-b644-4b2d-91c4-a68c01e2344d admin admin] PATH_INFO: `/v3/projects/48d87e6406724af3b1330e6f3cc15a28/users/870aa823b2e149dba5174fe9f7965a67/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:24.749310 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2cf08-659b-41d2-964a-5f5288d89480 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:24.754164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d829f21-a62e-4d0f-8db4-0a7039d1c2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:24.754825 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d829f21-a62e-4d0f-8db4-0a7039d1c2c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:24.755096 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d829f21-a62e-4d0f-8db4-0a7039d1c2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:24.755887 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d829f21-a62e-4d0f-8db4-0a7039d1c2c3 admin admin] PATH_INFO: `/v3/projects/66a33a727f9c40b99fb14bd2e0fc5090/users/db34ffc8f90247daac1548bbb421b015/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:24.768904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca7288-b644-4b2d-91c4-a68c01e2344d admin admin] RBAC: Authorizing `identity:create_grant(project_id=48d87e6406724af3b1330e6f3cc15a28, user_id=870aa823b2e149dba5174fe9f7965a67, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:24.772878 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca7288-b644-4b2d-91c4-a68c01e2344d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:24.777199 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d829f21-a62e-4d0f-8db4-0a7039d1c2c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66a33a727f9c40b99fb14bd2e0fc5090, user_id=db34ffc8f90247daac1548bbb421b015, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:24.780047 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d829f21-a62e-4d0f-8db4-0a7039d1c2c3 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:24.795372 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 640/1278] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:04:24 2022] PUT /identity/v3/projects/48d87e6406724af3b1330e6f3cc15a28/users/870aa823b2e149dba5174fe9f7965a67/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:04:24.805817 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca7288-b644-4b2d-91c4-a68c01e2344d admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:24.805958 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 639/1279] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:04:24 2022] PUT /identity/v3/projects/66a33a727f9c40b99fb14bd2e0fc5090/users/db34ffc8f90247daac1548bbb421b015/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:04:24.805958 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d829f21-a62e-4d0f-8db4-0a7039d1c2c3 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:24.838099 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:24.888294 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca7288-b644-4b2d-91c4-a68c01e2344d admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:24.918959 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d829f21-a62e-4d0f-8db4-0a7039d1c2c3 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:24.929985 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0dbd8e-5332-469a-b50f-8449137e76f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:24.930676 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0dbd8e-5332-469a-b50f-8449137e76f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:24.930949 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0dbd8e-5332-469a-b50f-8449137e76f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:24.931221 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0dbd8e-5332-469a-b50f-8449137e76f5 admin admin] PATH_INFO: `/v3/projects/7a0d68c2055e47eaaca5f05313aff446/users/f8b6fcf434264a7c8f98840220ec4b03/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:24.948654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0dbd8e-5332-469a-b50f-8449137e76f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a0d68c2055e47eaaca5f05313aff446, user_id=f8b6fcf434264a7c8f98840220ec4b03, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:24.951601 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0dbd8e-5332-469a-b50f-8449137e76f5 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:24.970345 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 640/1280] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:04:24 2022] PUT /identity/v3/projects/7a0d68c2055e47eaaca5f05313aff446/users/f8b6fcf434264a7c8f98840220ec4b03/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:04:24.971556 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0dbd8e-5332-469a-b50f-8449137e76f5 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:24.973889 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e5906d-7f06-4366-ae1b-cfcc282831be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:24.975961 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5906d-7f06-4366-ae1b-cfcc282831be admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:24.976244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5906d-7f06-4366-ae1b-cfcc282831be admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:24.976522 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5906d-7f06-4366-ae1b-cfcc282831be admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:24.977208 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e5906d-7f06-4366-ae1b-cfcc282831be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:24.978882 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e5906d-7f06-4366-ae1b-cfcc282831be admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:24.995835 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 641/1281] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:24.997395 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d398c6-4068-4510-9f57-e06b37b5e3c2 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:24.997796 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d398c6-4068-4510-9f57-e06b37b5e3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:24.998062 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d398c6-4068-4510-9f57-e06b37b5e3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:25.019771 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-96d398c6-4068-4510-9f57-e06b37b5e3c2 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:25.021028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-96d398c6-4068-4510-9f57-e06b37b5e3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:25.078582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0dbd8e-5332-469a-b50f-8449137e76f5 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:25.084764 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab74f23-6a3d-40d3-8c80-b1a813aad586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:25.085395 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab74f23-6a3d-40d3-8c80-b1a813aad586 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:25.091470 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 642/1282] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:25.092112 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d398c6-4068-4510-9f57-e06b37b5e3c2 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:25.092878 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab74f23-6a3d-40d3-8c80-b1a813aad586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:25.093171 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab74f23-6a3d-40d3-8c80-b1a813aad586 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:25.093845 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab74f23-6a3d-40d3-8c80-b1a813aad586 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:25.095957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab74f23-6a3d-40d3-8c80-b1a813aad586 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:25.104240 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d398c6-4068-4510-9f57-e06b37b5e3c2 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:25.158290 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089bab2-a573-41fb-8b7a-77f92e9723be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:25.158290 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089bab2-a573-41fb-8b7a-77f92e9723be admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:25.158290 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089bab2-a573-41fb-8b7a-77f92e9723be admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:25.158290 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089bab2-a573-41fb-8b7a-77f92e9723be admin admin] PATH_INFO: `/v3/projects/72e1785381dd4bdaa2803fa7ef711dcb/users/c85953d8e7394fb0a6f425ce51420325/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:25.159171 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 641/1283] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:04:24 2022] POST /identity/v3/projects => generated 365 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:25.159171 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cde327-d90d-4fff-9850-f2d4eafd1fab None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:25.159171 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cde327-d90d-4fff-9850-f2d4eafd1fab None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:25.159171 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cde327-d90d-4fff-9850-f2d4eafd1fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:25.159799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089bab2-a573-41fb-8b7a-77f92e9723be admin admin] RBAC: Authorizing `identity:create_grant(project_id=72e1785381dd4bdaa2803fa7ef711dcb, user_id=c85953d8e7394fb0a6f425ce51420325, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:25.159799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089bab2-a573-41fb-8b7a-77f92e9723be admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:25.170613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:25.182964 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 643/1284] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:04:25 2022] PUT /identity/v3/projects/72e1785381dd4bdaa2803fa7ef711dcb/users/c85953d8e7394fb0a6f425ce51420325/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:04:25.187654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-07cde327-d90d-4fff-9850-f2d4eafd1fab None None] MFA Rules not processed for user `870aa823b2e149dba5174fe9f7965a67`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:25.188652 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c8dc4-2273-41cc-9cbd-7c33ba57af3e None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:25.188962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c8dc4-2273-41cc-9cbd-7c33ba57af3e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:25.189249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-07cde327-d90d-4fff-9850-f2d4eafd1fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:25.189505 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c8dc4-2273-41cc-9cbd-7c33ba57af3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:25.223446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d94c8dc4-2273-41cc-9cbd-7c33ba57af3e None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:25.225981 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d94c8dc4-2273-41cc-9cbd-7c33ba57af3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:25.275409 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 642/1285] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:25.281111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19800502-f4ad-481f-ba22-bb575766fb7f None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:25.281371 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:25.281855 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19800502-f4ad-481f-ba22-bb575766fb7f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:25.281855 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19800502-f4ad-481f-ba22-bb575766fb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:25.325237 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-19800502-f4ad-481f-ba22-bb575766fb7f None None] MFA Rules not processed for user `f8b6fcf434264a7c8f98840220ec4b03`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:25.326551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 644/1286] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:25.326785 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-19800502-f4ad-481f-ba22-bb575766fb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:25.327459 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c8dc4-2273-41cc-9cbd-7c33ba57af3e None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:25.340525 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c8dc4-2273-41cc-9cbd-7c33ba57af3e None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:25.346449 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e68a5-c1a5-4e9a-88fb-ba4408600791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:25.347254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e68a5-c1a5-4e9a-88fb-ba4408600791 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:25.347536 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e68a5-c1a5-4e9a-88fb-ba4408600791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:25.347805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e68a5-c1a5-4e9a-88fb-ba4408600791 admin admin] PATH_INFO: `/v3/projects/b21953597b4c45059f7f78ac8fdecbee/users/6da8a7cb6ef849faad7d7091a1095ab9/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:25.364195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e68a5-c1a5-4e9a-88fb-ba4408600791 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b21953597b4c45059f7f78ac8fdecbee, user_id=6da8a7cb6ef849faad7d7091a1095ab9, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:25.367186 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e68a5-c1a5-4e9a-88fb-ba4408600791 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:25.381648 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 645/1287] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:04:25 2022] PUT /identity/v3/projects/b21953597b4c45059f7f78ac8fdecbee/users/6da8a7cb6ef849faad7d7091a1095ab9/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:04:25.382500 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e68a5-c1a5-4e9a-88fb-ba4408600791 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:25.405903 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 643/1288] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:25.406696 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19800502-f4ad-481f-ba22-bb575766fb7f None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:25.418952 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19800502-f4ad-481f-ba22-bb575766fb7f None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:25.444392 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e68a5-c1a5-4e9a-88fb-ba4408600791 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:25.448751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7505bae-f37f-4c62-a13d-630d2f4f4c4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:25.449345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7505bae-f37f-4c62-a13d-630d2f4f4c4d service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:25.449619 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7505bae-f37f-4c62-a13d-630d2f4f4c4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:25.449888 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7505bae-f37f-4c62-a13d-630d2f4f4c4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:25.497377 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0e42e-5a01-47ca-bcee-d4e24d94b3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:25.498159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0e42e-5a01-47ca-bcee-d4e24d94b3e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:25.498624 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0e42e-5a01-47ca-bcee-d4e24d94b3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:25.499023 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0e42e-5a01-47ca-bcee-d4e24d94b3e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:25.500131 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0e42e-5a01-47ca-bcee-d4e24d94b3e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:25.502838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0e42e-5a01-47ca-bcee-d4e24d94b3e7 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:25.520362 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7505bae-f37f-4c62-a13d-630d2f4f4c4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:25.522401 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7505bae-f37f-4c62-a13d-630d2f4f4c4d service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:25.532571 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 644/1289] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:04:25 2022] POST /identity/v3/users => generated 289 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:25.532571 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac90c40-cf38-4f20-b70b-8f19766cbc52 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:25.532571 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac90c40-cf38-4f20-b70b-8f19766cbc52 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:25.532571 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac90c40-cf38-4f20-b70b-8f19766cbc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:25.539067 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 646/1290] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:04:25.540348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f6d2a-e369-44a3-b87f-581e2f1b55e9 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:25.540643 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f6d2a-e369-44a3-b87f-581e2f1b55e9 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:25.540938 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f6d2a-e369-44a3-b87f-581e2f1b55e9 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:25.542499 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 647/1291] 10.222.0.20 () {54 vars in 961 bytes} [Thu Oct 20 06:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 06:04:25.543204 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f6d2a-e369-44a3-b87f-581e2f1b55e9 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:25.555611 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-fac90c40-cf38-4f20-b70b-8f19766cbc52 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:25.555839 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f6d2a-e369-44a3-b87f-581e2f1b55e9 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:25.557094 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fac90c40-cf38-4f20-b70b-8f19766cbc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:25.560011 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ac7c4-f44c-4f45-99bd-5f12d7686cad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:25.560787 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ac7c4-f44c-4f45-99bd-5f12d7686cad service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:25.561071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ac7c4-f44c-4f45-99bd-5f12d7686cad service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:25.561345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ac7c4-f44c-4f45-99bd-5f12d7686cad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:25.567233 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 645/1292] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:25.569034 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521df7e-1a96-4162-b2bf-56ac88c91503 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:25.569528 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521df7e-1a96-4162-b2bf-56ac88c91503 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:25.570080 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521df7e-1a96-4162-b2bf-56ac88c91503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:25.574904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ac7c4-f44c-4f45-99bd-5f12d7686cad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:25.576997 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ac7c4-f44c-4f45-99bd-5f12d7686cad service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:25.589493 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 648/1293] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:04:25.590475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ac7c4-f44c-4f45-99bd-5f12d7686cad service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:25.599915 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-d521df7e-1a96-4162-b2bf-56ac88c91503 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:25.603175 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d521df7e-1a96-4162-b2bf-56ac88c91503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:25.607124 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ac7c4-f44c-4f45-99bd-5f12d7686cad service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:25.614438 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 646/1294] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:25.614438 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521df7e-1a96-4162-b2bf-56ac88c91503 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:25.616838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78daaf6e-00fd-46f4-aadf-ff9762a075f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:25.617466 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78daaf6e-00fd-46f4-aadf-ff9762a075f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:25.619498 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78daaf6e-00fd-46f4-aadf-ff9762a075f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:25.619834 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78daaf6e-00fd-46f4-aadf-ff9762a075f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:25.620483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78daaf6e-00fd-46f4-aadf-ff9762a075f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:25.628040 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78daaf6e-00fd-46f4-aadf-ff9762a075f1 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:25.633017 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521df7e-1a96-4162-b2bf-56ac88c91503 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:25.649205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5bad6d-5f01-4038-8c45-65af6543ae2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:25.649962 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5bad6d-5f01-4038-8c45-65af6543ae2f admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:25.650340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5bad6d-5f01-4038-8c45-65af6543ae2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:25.650653 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5bad6d-5f01-4038-8c45-65af6543ae2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:25.651397 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5bad6d-5f01-4038-8c45-65af6543ae2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:25.660867 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5bad6d-5f01-4038-8c45-65af6543ae2f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:25.665407 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:25.694642 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 649/1295] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:04:25 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:25.694879 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 647/1296] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:25.694879 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5bad6d-5f01-4038-8c45-65af6543ae2f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:25.695074 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7f975-5331-43c3-9a97-050dbdf615cd None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:25.695074 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7f975-5331-43c3-9a97-050dbdf615cd None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:25.695074 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7f975-5331-43c3-9a97-050dbdf615cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:25.695363 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3f5bad6d-5f01-4038-8c45-65af6543ae2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:25.720954 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5bad6d-5f01-4038-8c45-65af6543ae2f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:25.748622 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-96b7f975-5331-43c3-9a97-050dbdf615cd None None] MFA Rules not processed for user `ae2c1c886cd94d9f90082f66e7e8c9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:25.750227 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-96b7f975-5331-43c3-9a97-050dbdf615cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:25.777742 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:25.825638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ae135-42e7-4e82-a136-babcd35ae262 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:25.826853 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ae135-42e7-4e82-a136-babcd35ae262 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:25.827349 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ae135-42e7-4e82-a136-babcd35ae262 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:25.827782 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ae135-42e7-4e82-a136-babcd35ae262 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:25.891565 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 650/1297] 10.222.0.20 () {58 vars in 1058 bytes} [Thu Oct 20 06:04:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:25.892421 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7f975-5331-43c3-9a97-050dbdf615cd None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:25.937925 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ae135-42e7-4e82-a136-babcd35ae262 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:25.939964 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ae135-42e7-4e82-a136-babcd35ae262 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:25.956281 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 648/1298] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:04:25.959143 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ae135-42e7-4e82-a136-babcd35ae262 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:25.983253 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ae135-42e7-4e82-a136-babcd35ae262 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:25.997145 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318d858-373b-40bc-913a-3de344618731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:25.998097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318d858-373b-40bc-913a-3de344618731 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:25.998215 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318d858-373b-40bc-913a-3de344618731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:25.998505 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318d858-373b-40bc-913a-3de344618731 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:26.006005 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b318d858-373b-40bc-913a-3de344618731 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:26.008060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b318d858-373b-40bc-913a-3de344618731 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:26.022794 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7f975-5331-43c3-9a97-050dbdf615cd None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:26.035425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6a1bc-c2a1-4815-8f7c-dd1e4d91fcf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:26.036796 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6a1bc-c2a1-4815-8f7c-dd1e4d91fcf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:26.037581 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6a1bc-c2a1-4815-8f7c-dd1e4d91fcf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:26.038297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6a1bc-c2a1-4815-8f7c-dd1e4d91fcf4 admin admin] PATH_INFO: `/v3/projects/4cf38fcec6b140d8adbbb94ba4d2fc1a/users/b4acf4f64a67401a8fc1336bd59b592f/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:26.055631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 649/1299] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:04:25 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:26.056802 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318d858-373b-40bc-913a-3de344618731 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:26.077688 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318d858-373b-40bc-913a-3de344618731 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:26.084020 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94e943-e31e-4f84-af38-0ca3bc3a062b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:26.085334 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94e943-e31e-4f84-af38-0ca3bc3a062b service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:26.085730 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94e943-e31e-4f84-af38-0ca3bc3a062b service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:26.086896 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94e943-e31e-4f84-af38-0ca3bc3a062b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:26.103818 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94e943-e31e-4f84-af38-0ca3bc3a062b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:26.107431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94e943-e31e-4f84-af38-0ca3bc3a062b service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:26.109582 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae6a1bc-c2a1-4815-8f7c-dd1e4d91fcf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cf38fcec6b140d8adbbb94ba4d2fc1a, user_id=b4acf4f64a67401a8fc1336bd59b592f, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:26.112826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae6a1bc-c2a1-4815-8f7c-dd1e4d91fcf4 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:26.124175 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 650/1300] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:04:26.125425 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94e943-e31e-4f84-af38-0ca3bc3a062b service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:26.134446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 651/1301] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:04:25 2022] PUT /identity/v3/projects/4cf38fcec6b140d8adbbb94ba4d2fc1a/users/b4acf4f64a67401a8fc1336bd59b592f/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:04:26.135414 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6a1bc-c2a1-4815-8f7c-dd1e4d91fcf4 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:26.144944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94e943-e31e-4f84-af38-0ca3bc3a062b service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:26.269305 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6a1bc-c2a1-4815-8f7c-dd1e4d91fcf4 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:26.283158 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c79b6-6fa3-4923-8c65-eda756e483f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:26.285026 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c79b6-6fa3-4923-8c65-eda756e483f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:26.285361 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c79b6-6fa3-4923-8c65-eda756e483f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:26.286172 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c79b6-6fa3-4923-8c65-eda756e483f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:26.288799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c79b6-6fa3-4923-8c65-eda756e483f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:26.291596 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c79b6-6fa3-4923-8c65-eda756e483f5 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:26.292987 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da80a5d-b1ab-4379-8693-25b661f2267d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:26.294846 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da80a5d-b1ab-4379-8693-25b661f2267d service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:26.297304 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da80a5d-b1ab-4379-8693-25b661f2267d service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:26.297304 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da80a5d-b1ab-4379-8693-25b661f2267d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:26.320279 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 652/1302] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:26.324078 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c79b6-6fa3-4923-8c65-eda756e483f5 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:26.326253 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:26.364347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c79b6-6fa3-4923-8c65-eda756e483f5 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:26.378938 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad5392-17f4-4a6f-822a-ec77cecc5c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:26.379649 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad5392-17f4-4a6f-822a-ec77cecc5c56 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:26.381311 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad5392-17f4-4a6f-822a-ec77cecc5c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:26.381776 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad5392-17f4-4a6f-822a-ec77cecc5c56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:26.383672 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ad5392-17f4-4a6f-822a-ec77cecc5c56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:26.389682 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ad5392-17f4-4a6f-822a-ec77cecc5c56 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:26.418781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 653/1303] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:26.425595 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad5392-17f4-4a6f-822a-ec77cecc5c56 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:26.462221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad5392-17f4-4a6f-822a-ec77cecc5c56 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:26.478847 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceecb1c8-8349-4cda-a60f-e145e0115ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:26.480160 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceecb1c8-8349-4cda-a60f-e145e0115ba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:26.480510 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceecb1c8-8349-4cda-a60f-e145e0115ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:26.480873 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceecb1c8-8349-4cda-a60f-e145e0115ba7 admin admin] PATH_INFO: `/v3/projects/0a8a9d59285e473fba9b05d851917eda/users/ca89311f940b42ac9104cbc6f1e52f7f/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:26.491782 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da80a5d-b1ab-4379-8693-25b661f2267d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:26.493472 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da80a5d-b1ab-4379-8693-25b661f2267d service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:26.526562 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceecb1c8-8349-4cda-a60f-e145e0115ba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a8a9d59285e473fba9b05d851917eda, user_id=ca89311f940b42ac9104cbc6f1e52f7f, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:26.527278 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 651/1304] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:04:26.528124 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da80a5d-b1ab-4379-8693-25b661f2267d service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:26.534448 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceecb1c8-8349-4cda-a60f-e145e0115ba7 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:26.555629 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da80a5d-b1ab-4379-8693-25b661f2267d service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:26.558705 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:26.569385 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 654/1305] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:04:26 2022] PUT /identity/v3/projects/0a8a9d59285e473fba9b05d851917eda/users/ca89311f940b42ac9104cbc6f1e52f7f/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:04:26.573495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceecb1c8-8349-4cda-a60f-e145e0115ba7 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:26.573913 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1e318-877b-419b-96f2-a00604f55204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:26.574617 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1e318-877b-419b-96f2-a00604f55204 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:26.574901 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1e318-877b-419b-96f2-a00604f55204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:26.575656 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1e318-877b-419b-96f2-a00604f55204 admin admin] PATH_INFO: `/v3/projects/4cf38fcec6b140d8adbbb94ba4d2fc1a/users/b4acf4f64a67401a8fc1336bd59b592f/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:26.582874 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac1e318-877b-419b-96f2-a00604f55204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cf38fcec6b140d8adbbb94ba4d2fc1a, user_id=b4acf4f64a67401a8fc1336bd59b592f, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:26.586264 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac1e318-877b-419b-96f2-a00604f55204 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:26.622990 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 652/1306] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:04:26 2022] PUT /identity/v3/projects/4cf38fcec6b140d8adbbb94ba4d2fc1a/users/b4acf4f64a67401a8fc1336bd59b592f/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:04:26.624564 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc613d5f-18ef-4708-8b03-f0a1f668e3b6 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:26.624885 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc613d5f-18ef-4708-8b03-f0a1f668e3b6 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:26.625168 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc613d5f-18ef-4708-8b03-f0a1f668e3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:26.694500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-fc613d5f-18ef-4708-8b03-f0a1f668e3b6 None None] MFA Rules not processed for user `b4acf4f64a67401a8fc1336bd59b592f`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:26.695576 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fc613d5f-18ef-4708-8b03-f0a1f668e3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:26.743478 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceecb1c8-8349-4cda-a60f-e145e0115ba7 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:26.874236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 653/1307] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:26.883214 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc613d5f-18ef-4708-8b03-f0a1f668e3b6 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:26.914041 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65883a2-8794-4798-a578-6200a7a0b39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:26.917997 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65883a2-8794-4798-a578-6200a7a0b39c admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:26.917997 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65883a2-8794-4798-a578-6200a7a0b39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:26.918248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65883a2-8794-4798-a578-6200a7a0b39c admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:26.919783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65883a2-8794-4798-a578-6200a7a0b39c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:26.924832 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65883a2-8794-4798-a578-6200a7a0b39c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:26.954137 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 655/1308] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:26.957738 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65883a2-8794-4798-a578-6200a7a0b39c admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:26.990889 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65883a2-8794-4798-a578-6200a7a0b39c admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:27.006940 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ecf6d-20c3-48a2-ade2-4b226ef85a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:27.010984 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ecf6d-20c3-48a2-ade2-4b226ef85a5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:27.011485 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ecf6d-20c3-48a2-ade2-4b226ef85a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:27.011961 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ecf6d-20c3-48a2-ade2-4b226ef85a5a admin admin] PATH_INFO: `/v3/projects/0a8a9d59285e473fba9b05d851917eda/users/ca89311f940b42ac9104cbc6f1e52f7f/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:27.013458 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc613d5f-18ef-4708-8b03-f0a1f668e3b6 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:27.017791 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ecf6d-20c3-48a2-ade2-4b226ef85a5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a8a9d59285e473fba9b05d851917eda, user_id=ca89311f940b42ac9104cbc6f1e52f7f, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:27.024943 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ecf6d-20c3-48a2-ade2-4b226ef85a5a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:27.025414 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd7d57-7baa-4e05-bacf-5ea32fae8e09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:27.026052 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd7d57-7baa-4e05-bacf-5ea32fae8e09 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:27.026338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd7d57-7baa-4e05-bacf-5ea32fae8e09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:27.026731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd7d57-7baa-4e05-bacf-5ea32fae8e09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:27.047221 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd7d57-7baa-4e05-bacf-5ea32fae8e09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:27.048723 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd7d57-7baa-4e05-bacf-5ea32fae8e09 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:27.058889 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 656/1309] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:04:26 2022] PUT /identity/v3/projects/0a8a9d59285e473fba9b05d851917eda/users/ca89311f940b42ac9104cbc6f1e52f7f/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:04:27.070721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386af16-cbed-4cbc-9a1b-6b27bf134f32 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:27.071029 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386af16-cbed-4cbc-9a1b-6b27bf134f32 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:27.071311 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386af16-cbed-4cbc-9a1b-6b27bf134f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:27.092449 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 654/1310] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:04:27.135205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477aa91-12a7-4070-95a3-33ac2ee48d0f None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:27.135567 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477aa91-12a7-4070-95a3-33ac2ee48d0f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:27.135822 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477aa91-12a7-4070-95a3-33ac2ee48d0f None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:27.137495 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 655/1311] 10.222.0.20 () {54 vars in 912 bytes} [Thu Oct 20 06:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 06:04:27.159569 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529eba00-ea7d-4566-941d-8d2c5c2583e5 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:27.161564 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529eba00-ea7d-4566-941d-8d2c5c2583e5 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:27.161564 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529eba00-ea7d-4566-941d-8d2c5c2583e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:27.172637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-c386af16-cbed-4cbc-9a1b-6b27bf134f32 None None] MFA Rules not processed for user `ca89311f940b42ac9104cbc6f1e52f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:27.174080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c386af16-cbed-4cbc-9a1b-6b27bf134f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:27.212346 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-529eba00-ea7d-4566-941d-8d2c5c2583e5 None None] MFA Rules not processed for user `ae2c1c886cd94d9f90082f66e7e8c9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:27.219113 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-529eba00-ea7d-4566-941d-8d2c5c2583e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:27.313471 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:27.417647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 657/1312] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:27.419175 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386af16-cbed-4cbc-9a1b-6b27bf134f32 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:27.436213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 656/1313] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Oct 20 06:04:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:27.437333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529eba00-ea7d-4566-941d-8d2c5c2583e5 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:27.449265 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:27.546359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386af16-cbed-4cbc-9a1b-6b27bf134f32 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:27.556723 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8c269-202b-4f38-aba6-93a16b3a5f4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:27.557773 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8c269-202b-4f38-aba6-93a16b3a5f4f service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:27.557934 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8c269-202b-4f38-aba6-93a16b3a5f4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:27.558181 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8c269-202b-4f38-aba6-93a16b3a5f4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:27.571741 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529eba00-ea7d-4566-941d-8d2c5c2583e5 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:27.581182 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad7e66e-b5a2-440f-9315-673f604a258d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:27.581817 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7e66e-b5a2-440f-9315-673f604a258d service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:27.582092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7e66e-b5a2-440f-9315-673f604a258d service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:27.582358 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7e66e-b5a2-440f-9315-673f604a258d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:27.596945 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7e66e-b5a2-440f-9315-673f604a258d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:27.600934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7e66e-b5a2-440f-9315-673f604a258d service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:27.620252 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 657/1314] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:04:27.661057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8c269-202b-4f38-aba6-93a16b3a5f4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:27.664640 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8c269-202b-4f38-aba6-93a16b3a5f4f service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:27.689815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 658/1315] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:04:27.722408 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad7e66e-b5a2-440f-9315-673f604a258d service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:27.744030 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad7e66e-b5a2-440f-9315-673f604a258d service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:27.754874 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0fa29-a143-48e9-b960-dc148f18f53e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:27.757135 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0fa29-a143-48e9-b960-dc148f18f53e service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:27.758249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0fa29-a143-48e9-b960-dc148f18f53e service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:27.758745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0fa29-a143-48e9-b960-dc148f18f53e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:27.776558 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a0fa29-a143-48e9-b960-dc148f18f53e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:27.779503 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a0fa29-a143-48e9-b960-dc148f18f53e service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:27.799578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 658/1316] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:04:28.335624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8c269-202b-4f38-aba6-93a16b3a5f4f service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:28.345981 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8c269-202b-4f38-aba6-93a16b3a5f4f service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:28.353898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d065507-1824-4baa-b1c3-82a185a31aab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:28.354837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d065507-1824-4baa-b1c3-82a185a31aab service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:28.355257 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d065507-1824-4baa-b1c3-82a185a31aab service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:28.357001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d065507-1824-4baa-b1c3-82a185a31aab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:28.529267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d065507-1824-4baa-b1c3-82a185a31aab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:28.532527 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d065507-1824-4baa-b1c3-82a185a31aab service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:28.558781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 659/1317] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:04:29.164334 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d5319-cc97-40cf-b0ae-4ce0cb449cb2 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:29.164875 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d5319-cc97-40cf-b0ae-4ce0cb449cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:29.165129 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d5319-cc97-40cf-b0ae-4ce0cb449cb2 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:29.167241 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 659/1318] 10.222.0.20 () {54 vars in 961 bytes} [Thu Oct 20 06:04:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 06:04:29.177970 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058fe828-104b-4183-9252-5c640eb0910f None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:29.178462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058fe828-104b-4183-9252-5c640eb0910f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:29.178575 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058fe828-104b-4183-9252-5c640eb0910f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:29.206659 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-058fe828-104b-4183-9252-5c640eb0910f None None] MFA Rules not processed for user `ae2c1c886cd94d9f90082f66e7e8c9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:29.208359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-058fe828-104b-4183-9252-5c640eb0910f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:29.230917 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 660/1319] 10.222.0.20 () {58 vars in 1058 bytes} [Thu Oct 20 06:04:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:29.232558 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d982df7-c1dd-4171-81cb-a11f4097d813 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:29.233028 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d982df7-c1dd-4171-81cb-a11f4097d813 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:29.234428 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d982df7-c1dd-4171-81cb-a11f4097d813 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:29.235934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 660/1320] 10.222.0.20 () {54 vars in 912 bytes} [Thu Oct 20 06:04:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 06:04:29.244183 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f17c97-3658-48e0-a3a5-bf44494b9c1d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:29.244599 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f17c97-3658-48e0-a3a5-bf44494b9c1d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:29.245044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f17c97-3658-48e0-a3a5-bf44494b9c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:29.280645 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-65f17c97-3658-48e0-a3a5-bf44494b9c1d None None] MFA Rules not processed for user `ae2c1c886cd94d9f90082f66e7e8c9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:29.281511 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-65f17c97-3658-48e0-a3a5-bf44494b9c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:29.305185 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 661/1321] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Oct 20 06:04:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:29.319266 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d982df7-c1dd-4171-81cb-a11f4097d813 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:29.351121 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d982df7-c1dd-4171-81cb-a11f4097d813 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:29.365260 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f46be-cb1d-4e4f-ae16-b580de265616 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:29.366913 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f46be-cb1d-4e4f-ae16-b580de265616 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:29.367050 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f46be-cb1d-4e4f-ae16-b580de265616 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:29.367326 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f46be-cb1d-4e4f-ae16-b580de265616 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:29.404542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f46be-cb1d-4e4f-ae16-b580de265616 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:29.416151 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f46be-cb1d-4e4f-ae16-b580de265616 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:29.460635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 661/1322] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:04:32.180801 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5a0dd-d616-4224-99d0-ed0ff4a7dea5 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:32.181334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5a0dd-d616-4224-99d0-ed0ff4a7dea5 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:32.181702 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5a0dd-d616-4224-99d0-ed0ff4a7dea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:32.275031 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a2f5a0dd-d616-4224-99d0-ed0ff4a7dea5 None None] MFA Rules not processed for user `2c6be23065c643939055a3de375187d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:32.279011 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a2f5a0dd-d616-4224-99d0-ed0ff4a7dea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:32.422540 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:32.548627 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 662/1323] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:32.565224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f46be-cb1d-4e4f-ae16-b580de265616 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:32.599713 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f46be-cb1d-4e4f-ae16-b580de265616 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:32.621213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce46821-3122-40e0-98c8-41882266f0a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:32.623644 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce46821-3122-40e0-98c8-41882266f0a9 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:32.624434 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce46821-3122-40e0-98c8-41882266f0a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:32.625751 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce46821-3122-40e0-98c8-41882266f0a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:32.659093 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce46821-3122-40e0-98c8-41882266f0a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:32.665630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce46821-3122-40e0-98c8-41882266f0a9 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:32.702718 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 662/1324] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:04:38.608011 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c1f9d-952a-4fcc-b9f2-9747d74d8356 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:38.608506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c1f9d-952a-4fcc-b9f2-9747d74d8356 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:38.608594 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c1f9d-952a-4fcc-b9f2-9747d74d8356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:38.703450 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-df7c1f9d-952a-4fcc-b9f2-9747d74d8356 None None] MFA Rules not processed for user `57d4cbd2a2154e5d925462df7c77815a`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:38.705850 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-df7c1f9d-952a-4fcc-b9f2-9747d74d8356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:38.898649 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 663/1325] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:38 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:38.909597 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adacf284-6f42-4355-8994-ba152b809f40 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:38.910228 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adacf284-6f42-4355-8994-ba152b809f40 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:38.910228 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adacf284-6f42-4355-8994-ba152b809f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:38.946498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-adacf284-6f42-4355-8994-ba152b809f40 None None] MFA Rules not processed for user `57d4cbd2a2154e5d925462df7c77815a`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:38.948780 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-adacf284-6f42-4355-8994-ba152b809f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:38.992793 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 663/1326] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:38 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:39.004319 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c1f9d-952a-4fcc-b9f2-9747d74d8356 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:39.034932 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c1f9d-952a-4fcc-b9f2-9747d74d8356 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:39.049647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e2449-f145-4fff-9cbe-0c41d66b730e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:39.049995 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e2449-f145-4fff-9cbe-0c41d66b730e service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:39.051179 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e2449-f145-4fff-9cbe-0c41d66b730e service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:39.051466 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e2449-f145-4fff-9cbe-0c41d66b730e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:39.072173 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e2449-f145-4fff-9cbe-0c41d66b730e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:39.073916 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e2449-f145-4fff-9cbe-0c41d66b730e service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:39.085689 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 664/1327] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:39 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:04:40.115970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a106c-f291-43c7-bf39-624b27a4d8c5 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:40.117011 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a106c-f291-43c7-bf39-624b27a4d8c5 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:40.117477 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a106c-f291-43c7-bf39-624b27a4d8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:40.171512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-759a106c-f291-43c7-bf39-624b27a4d8c5 None None] MFA Rules not processed for user `db34ffc8f90247daac1548bbb421b015`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:40.174535 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-759a106c-f291-43c7-bf39-624b27a4d8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:40.221708 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a16d6-d0c6-4c36-ad21-6370d61eac41 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:40.222616 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a16d6-d0c6-4c36-ad21-6370d61eac41 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:40.223104 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a16d6-d0c6-4c36-ad21-6370d61eac41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:40.252911 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-204a16d6-d0c6-4c36-ad21-6370d61eac41 None None] MFA Rules not processed for user `2c6be23065c643939055a3de375187d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:40.255413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-204a16d6-d0c6-4c36-ad21-6370d61eac41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:40.273615 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 665/1328] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:40.305357 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 664/1329] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:40.559227 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebfea6-16d7-4b7b-9680-2ad1c9658e61 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:40.559664 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebfea6-16d7-4b7b-9680-2ad1c9658e61 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:40.560053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebfea6-16d7-4b7b-9680-2ad1c9658e61 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:40.561567 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 666/1330] 10.222.0.20 () {54 vars in 912 bytes} [Thu Oct 20 06:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 06:04:40.568018 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f9416-7c49-4157-828f-32ec4114cd99 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:40.568522 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f9416-7c49-4157-828f-32ec4114cd99 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:40.568977 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f9416-7c49-4157-828f-32ec4114cd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:40.581029 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:40.657647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-6e5f9416-7c49-4157-828f-32ec4114cd99 None None] MFA Rules not processed for user `9072ce66e7174837aeb8c7898026fcce`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:40.659058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6e5f9416-7c49-4157-828f-32ec4114cd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:40.823860 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 665/1331] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Oct 20 06:04:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:40.854809 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebfea6-16d7-4b7b-9680-2ad1c9658e61 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:40.875723 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebfea6-16d7-4b7b-9680-2ad1c9658e61 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:40.898855 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7e86d-df82-421c-8505-15eab6627b19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:40.899621 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7e86d-df82-421c-8505-15eab6627b19 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:40.900033 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7e86d-df82-421c-8505-15eab6627b19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:40.900419 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7e86d-df82-421c-8505-15eab6627b19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:40.920516 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b7e86d-df82-421c-8505-15eab6627b19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:40.925810 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b7e86d-df82-421c-8505-15eab6627b19 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:40.953159 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 667/1332] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:04:41.329914 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a52cc-2738-4df1-87fa-05fbc4338813 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:41.330982 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a52cc-2738-4df1-87fa-05fbc4338813 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:41.331361 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a52cc-2738-4df1-87fa-05fbc4338813 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:41.332902 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 666/1333] 10.222.0.20 () {54 vars in 912 bytes} [Thu Oct 20 06:04:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 06:04:41.341011 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a3c63-3ea4-4119-b8ae-24456c62de0f None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:41.342661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a3c63-3ea4-4119-b8ae-24456c62de0f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:41.343167 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a3c63-3ea4-4119-b8ae-24456c62de0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:41.369274 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-5c5a3c63-3ea4-4119-b8ae-24456c62de0f None None] MFA Rules not processed for user `9072ce66e7174837aeb8c7898026fcce`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:41.370677 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5c5a3c63-3ea4-4119-b8ae-24456c62de0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:41.392511 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a52cc-2738-4df1-87fa-05fbc4338813 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:41.409677 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 668/1334] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Oct 20 06:04:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:41.421972 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a52cc-2738-4df1-87fa-05fbc4338813 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:41.427287 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e913d-8562-441a-bb04-22481617bf1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:41.428281 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e913d-8562-441a-bb04-22481617bf1b service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:41.428971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e913d-8562-441a-bb04-22481617bf1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:41.429477 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e913d-8562-441a-bb04-22481617bf1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:41.450180 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e913d-8562-441a-bb04-22481617bf1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:41.451825 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e913d-8562-441a-bb04-22481617bf1b service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:41.469005 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 667/1335] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:04:42.279604 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a3c63-3ea4-4119-b8ae-24456c62de0f None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:42.307536 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a3c63-3ea4-4119-b8ae-24456c62de0f None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:42.326752 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57685e47-2332-4bbe-9c3b-15609eca0661 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:42.329189 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57685e47-2332-4bbe-9c3b-15609eca0661 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:42.330743 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57685e47-2332-4bbe-9c3b-15609eca0661 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:42.331313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57685e47-2332-4bbe-9c3b-15609eca0661 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:42.388964 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57685e47-2332-4bbe-9c3b-15609eca0661 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:42.405978 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57685e47-2332-4bbe-9c3b-15609eca0661 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:42.430843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e913d-8562-441a-bb04-22481617bf1b service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:42.481829 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 669/1336] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:42 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:04:42.483282 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe0339-a12d-4e50-9dff-db70635a44e9 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:42.483573 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe0339-a12d-4e50-9dff-db70635a44e9 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:42.483850 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe0339-a12d-4e50-9dff-db70635a44e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:42.620277 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-67fe0339-a12d-4e50-9dff-db70635a44e9 None None] MFA Rules not processed for user `c85953d8e7394fb0a6f425ce51420325`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:42.626644 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-67fe0339-a12d-4e50-9dff-db70635a44e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:42.649136 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e913d-8562-441a-bb04-22481617bf1b service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:42.676550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc795ad3-30a9-4252-97c2-f64f879da0fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfbaaf32475a4c82a34948602d8f4e35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:42.687044 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc795ad3-30a9-4252-97c2-f64f879da0fc service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:42.687720 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc795ad3-30a9-4252-97c2-f64f879da0fc service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:42.688893 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc795ad3-30a9-4252-97c2-f64f879da0fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:42.694376 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-cc795ad3-30a9-4252-97c2-f64f879da0fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:42.831643 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc795ad3-30a9-4252-97c2-f64f879da0fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:42.836911 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc795ad3-30a9-4252-97c2-f64f879da0fc service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:42.882161 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 670/1337] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:42.883691 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b859cd-680b-45bb-960d-01e85e4b2339 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:42.885577 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b859cd-680b-45bb-960d-01e85e4b2339 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:42.886722 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b859cd-680b-45bb-960d-01e85e4b2339 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:42.891920 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 671/1338] 10.222.0.20 () {54 vars in 961 bytes} [Thu Oct 20 06:04:42 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 06:04:42.899876 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecf533-c9fc-4064-90bf-6e9576736b3b None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:42.902194 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecf533-c9fc-4064-90bf-6e9576736b3b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:42.902664 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecf533-c9fc-4064-90bf-6e9576736b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:42.921230 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 668/1339] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:04:42.922721 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ec7d6-6b4d-4ec7-a1cb-c76f1a997a21 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:42.923015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ec7d6-6b4d-4ec7-a1cb-c76f1a997a21 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:42.923287 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ec7d6-6b4d-4ec7-a1cb-c76f1a997a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:42.994614 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-27ecf533-c9fc-4064-90bf-6e9576736b3b None None] MFA Rules not processed for user `c85953d8e7394fb0a6f425ce51420325`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:42.995865 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-27ecf533-c9fc-4064-90bf-6e9576736b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:43.024359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-bb0ec7d6-6b4d-4ec7-a1cb-c76f1a997a21 None None] MFA Rules not processed for user `cfbaaf32475a4c82a34948602d8f4e35`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:43.031846 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-bb0ec7d6-6b4d-4ec7-a1cb-c76f1a997a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:43.055189 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 672/1340] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:43.071593 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 669/1341] 10.222.0.20 () {58 vars in 1058 bytes} [Thu Oct 20 06:04:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:43.081085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ecf533-c9fc-4064-90bf-6e9576736b3b None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:43.105843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ecf533-c9fc-4064-90bf-6e9576736b3b None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:43.110284 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:43.126097 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeba2af-b6a5-4770-b769-a056f4c31d21 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfbaaf32475a4c82a34948602d8f4e35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:43.126965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeba2af-b6a5-4770-b769-a056f4c31d21 service placement] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:43.127429 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeba2af-b6a5-4770-b769-a056f4c31d21 service placement] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:43.127853 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeba2af-b6a5-4770-b769-a056f4c31d21 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:43.154302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeba2af-b6a5-4770-b769-a056f4c31d21 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:43.155956 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeba2af-b6a5-4770-b769-a056f4c31d21 service placement] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:43.196008 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 673/1342] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:04:43.344072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ec7d6-6b4d-4ec7-a1cb-c76f1a997a21 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:43.369156 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ec7d6-6b4d-4ec7-a1cb-c76f1a997a21 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:43.405038 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e8cee-56ab-4ddd-b983-b3ab72ff211a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:43.405846 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e8cee-56ab-4ddd-b983-b3ab72ff211a service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:43.405957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e8cee-56ab-4ddd-b983-b3ab72ff211a service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:43.406197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e8cee-56ab-4ddd-b983-b3ab72ff211a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:43.451273 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504e8cee-56ab-4ddd-b983-b3ab72ff211a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:43.453683 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504e8cee-56ab-4ddd-b983-b3ab72ff211a service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:43.482707 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 670/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:04:43.491555 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792b660-214c-49c5-9668-a584924e83ed None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:43.492115 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792b660-214c-49c5-9668-a584924e83ed None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:43.492115 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792b660-214c-49c5-9668-a584924e83ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:43.549401 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-e792b660-214c-49c5-9668-a584924e83ed None None] MFA Rules not processed for user `57d4cbd2a2154e5d925462df7c77815a`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:43.553277 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e792b660-214c-49c5-9668-a584924e83ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:43.597637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 674/1344] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:44.450345 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb88dd-88ec-42e4-96ff-6feb27fe9d21 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:44.450952 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb88dd-88ec-42e4-96ff-6feb27fe9d21 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:44.451401 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb88dd-88ec-42e4-96ff-6feb27fe9d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:44.497870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3dbb88dd-88ec-42e4-96ff-6feb27fe9d21 None None] MFA Rules not processed for user `6da8a7cb6ef849faad7d7091a1095ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:44.499514 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3dbb88dd-88ec-42e4-96ff-6feb27fe9d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:44.613578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:44.685241 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 671/1345] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:04:45.942468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792b660-214c-49c5-9668-a584924e83ed None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:45.977140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792b660-214c-49c5-9668-a584924e83ed None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:45.990228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a6b30-bbaa-4631-bef4-93f3a8f3189e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:45.991743 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a6b30-bbaa-4631-bef4-93f3a8f3189e service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:45.992142 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a6b30-bbaa-4631-bef4-93f3a8f3189e service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:45.992484 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a6b30-bbaa-4631-bef4-93f3a8f3189e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:46.014301 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a6b30-bbaa-4631-bef4-93f3a8f3189e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:46.034870 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a6b30-bbaa-4631-bef4-93f3a8f3189e service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:46.069189 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 675/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:04:46.269808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb88dd-88ec-42e4-96ff-6feb27fe9d21 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:46.480803 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb88dd-88ec-42e4-96ff-6feb27fe9d21 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:46.497399 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e40adf-ba01-4e21-b065-b7528d2e9dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:46.498311 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e40adf-ba01-4e21-b065-b7528d2e9dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:46.498730 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e40adf-ba01-4e21-b065-b7528d2e9dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:46.499083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e40adf-ba01-4e21-b065-b7528d2e9dc2 admin admin] PATH_INFO: `/v3/users/870aa823b2e149dba5174fe9f7965a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:46.500473 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e40adf-ba01-4e21-b065-b7528d2e9dc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=870aa823b2e149dba5174fe9f7965a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:46.505626 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e40adf-ba01-4e21-b065-b7528d2e9dc2 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:46.641901 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-f6e40adf-ba01-4e21-b065-b7528d2e9dc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '870aa823b2e149dba5174fe9f7965a67'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:46.648548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-f6e40adf-ba01-4e21-b065-b7528d2e9dc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '870aa823b2e149dba5174fe9f7965a67'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:46.667372 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-f6e40adf-ba01-4e21-b065-b7528d2e9dc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '870aa823b2e149dba5174fe9f7965a67'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:46.684761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-f6e40adf-ba01-4e21-b065-b7528d2e9dc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '870aa823b2e149dba5174fe9f7965a67'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:46.694371 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-f6e40adf-ba01-4e21-b065-b7528d2e9dc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '870aa823b2e149dba5174fe9f7965a67'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:46.702301 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 672/1347] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:04:46 2022] DELETE /identity/v3/users/870aa823b2e149dba5174fe9f7965a67 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:04:46.707236 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15715272-863e-49d0-ba1b-7ce0812e4ff1 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:46.707415 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15715272-863e-49d0-ba1b-7ce0812e4ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:46.708648 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15715272-863e-49d0-ba1b-7ce0812e4ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:46.741089 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-15715272-863e-49d0-ba1b-7ce0812e4ff1 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:46.743154 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-15715272-863e-49d0-ba1b-7ce0812e4ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:46.936079 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 676/1348] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:04:46.999915 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e40adf-ba01-4e21-b065-b7528d2e9dc2 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:47.001971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f6e40adf-ba01-4e21-b065-b7528d2e9dc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:47.213586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e40adf-ba01-4e21-b065-b7528d2e9dc2 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:47.226668 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558a635-4dca-454b-ac89-e16ef2d101c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:47.227566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558a635-4dca-454b-ac89-e16ef2d101c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:47.227986 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558a635-4dca-454b-ac89-e16ef2d101c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:47.228442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558a635-4dca-454b-ac89-e16ef2d101c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:47.255840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558a635-4dca-454b-ac89-e16ef2d101c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:47.270163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558a635-4dca-454b-ac89-e16ef2d101c1 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:47.313932 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15715272-863e-49d0-ba1b-7ce0812e4ff1 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:47.318246 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-15715272-863e-49d0-ba1b-7ce0812e4ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:47.339640 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15715272-863e-49d0-ba1b-7ce0812e4ff1 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:47.343877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 673/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:04:47.348003 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f1dfd-bc4c-42ee-8d4f-e8e362daee13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:47.349581 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f1dfd-bc4c-42ee-8d4f-e8e362daee13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:47.350398 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f1dfd-bc4c-42ee-8d4f-e8e362daee13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:47.353206 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f1dfd-bc4c-42ee-8d4f-e8e362daee13 admin admin] PATH_INFO: `/v3/users/645232f827e544219e703c1a75551c89` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:47.355449 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f1dfd-bc4c-42ee-8d4f-e8e362daee13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=645232f827e544219e703c1a75551c89)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:47.357088 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: WARNING py.warnings [None req-461f1dfd-bc4c-42ee-8d4f-e8e362daee13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 06:04:47.357088 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: warnings.warn(msg) Oct 20 06:04:47.357088 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: Oct 20 06:04:47.359603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f1dfd-bc4c-42ee-8d4f-e8e362daee13 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:47.476378 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-461f1dfd-bc4c-42ee-8d4f-e8e362daee13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '645232f827e544219e703c1a75551c89'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:47.477107 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-461f1dfd-bc4c-42ee-8d4f-e8e362daee13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '645232f827e544219e703c1a75551c89'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:47.492191 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-461f1dfd-bc4c-42ee-8d4f-e8e362daee13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '645232f827e544219e703c1a75551c89'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:47.532245 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-461f1dfd-bc4c-42ee-8d4f-e8e362daee13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '645232f827e544219e703c1a75551c89'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:47.533081 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-461f1dfd-bc4c-42ee-8d4f-e8e362daee13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '645232f827e544219e703c1a75551c89'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:47.570262 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728ccf9-80f4-4543-bba5-187947970a3d None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:47.570592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 677/1350] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:04:47 2022] DELETE /identity/v3/users/645232f827e544219e703c1a75551c89 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:04:47.570718 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728ccf9-80f4-4543-bba5-187947970a3d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:47.575407 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728ccf9-80f4-4543-bba5-187947970a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:47.640268 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-a728ccf9-80f4-4543-bba5-187947970a3d None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:47.641789 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a728ccf9-80f4-4543-bba5-187947970a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:47.648183 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:47.786216 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 674/1351] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:47.941458 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f1dfd-bc4c-42ee-8d4f-e8e362daee13 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:47.943582 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-461f1dfd-bc4c-42ee-8d4f-e8e362daee13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:48.189152 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83079d98-7906-4a4f-b539-3dc9cb52a708 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:48.189754 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83079d98-7906-4a4f-b539-3dc9cb52a708 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:48.190409 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83079d98-7906-4a4f-b539-3dc9cb52a708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:48.267927 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-83079d98-7906-4a4f-b539-3dc9cb52a708 None None] MFA Rules not processed for user `7a59f547a20e48ee88e3e620ba1e87f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:48.269318 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-83079d98-7906-4a4f-b539-3dc9cb52a708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:48.447652 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 675/1352] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Oct 20 06:04:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:04:48.503390 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:48.532687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f1dfd-bc4c-42ee-8d4f-e8e362daee13 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:48.541788 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290103f-a97f-4dc8-9e81-cdf0912cb33e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:48.542497 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290103f-a97f-4dc8-9e81-cdf0912cb33e service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:48.542832 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290103f-a97f-4dc8-9e81-cdf0912cb33e service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:48.543194 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290103f-a97f-4dc8-9e81-cdf0912cb33e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:48.574049 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290103f-a97f-4dc8-9e81-cdf0912cb33e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:48.582060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290103f-a97f-4dc8-9e81-cdf0912cb33e service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:48.600321 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 678/1353] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 659 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:04:48.694193 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83079d98-7906-4a4f-b539-3dc9cb52a708 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:48.695651 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-83079d98-7906-4a4f-b539-3dc9cb52a708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:48.914861 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:48.917864 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83079d98-7906-4a4f-b539-3dc9cb52a708 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:48.945310 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396aa3e-ca18-4d72-9eb0-771d1c179a1e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeeab506ab5c4d6386fc899086f0abf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:48.947036 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396aa3e-ca18-4d72-9eb0-771d1c179a1e service swift] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:48.948248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396aa3e-ca18-4d72-9eb0-771d1c179a1e service swift] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:48.950630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396aa3e-ca18-4d72-9eb0-771d1c179a1e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:48.964625 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396aa3e-ca18-4d72-9eb0-771d1c179a1e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:48.967804 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396aa3e-ca18-4d72-9eb0-771d1c179a1e service swift] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:49.001556 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 676/1354] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Oct 20 06:04:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 06:04:49.029725 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddeadbb-5bd7-4419-8993-fd7eab93310a None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:49.030222 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddeadbb-5bd7-4419-8993-fd7eab93310a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:49.030732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddeadbb-5bd7-4419-8993-fd7eab93310a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:49.120390 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-8ddeadbb-5bd7-4419-8993-fd7eab93310a None None] MFA Rules not processed for user `db34ffc8f90247daac1548bbb421b015`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:49.130077 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8ddeadbb-5bd7-4419-8993-fd7eab93310a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:49.465258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 679/1355] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:50.716858 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396aa3e-ca18-4d72-9eb0-771d1c179a1e service swift] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:50.722220 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3396aa3e-ca18-4d72-9eb0-771d1c179a1e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:50.759366 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396aa3e-ca18-4d72-9eb0-771d1c179a1e service swift] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:50.789623 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1026e84-5efd-4e16-bccd-8f5e17a014ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:50.791319 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1026e84-5efd-4e16-bccd-8f5e17a014ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:50.792626 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1026e84-5efd-4e16-bccd-8f5e17a014ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:50.793441 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1026e84-5efd-4e16-bccd-8f5e17a014ae admin admin] PATH_INFO: `/v3/projects/48d87e6406724af3b1330e6f3cc15a28` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:50.800940 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1026e84-5efd-4e16-bccd-8f5e17a014ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=48d87e6406724af3b1330e6f3cc15a28)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:50.805179 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: WARNING py.warnings [None req-e1026e84-5efd-4e16-bccd-8f5e17a014ae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 06:04:50.805179 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: warnings.warn(msg) Oct 20 06:04:50.805179 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: Oct 20 06:04:50.806686 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1026e84-5efd-4e16-bccd-8f5e17a014ae admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:51.026886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e1026e84-5efd-4e16-bccd-8f5e17a014ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48d87e6406724af3b1330e6f3cc15a28', 'request_id': 'req-e1026e84-5efd-4e16-bccd-8f5e17a014ae'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:51.037071 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e1026e84-5efd-4e16-bccd-8f5e17a014ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48d87e6406724af3b1330e6f3cc15a28', 'request_id': 'req-e1026e84-5efd-4e16-bccd-8f5e17a014ae'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:51.065647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e1026e84-5efd-4e16-bccd-8f5e17a014ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48d87e6406724af3b1330e6f3cc15a28', 'request_id': 'req-e1026e84-5efd-4e16-bccd-8f5e17a014ae'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:51.091531 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e1026e84-5efd-4e16-bccd-8f5e17a014ae admin admin] The token cache is being invalidate because project 48d87e6406724af3b1330e6f3cc15a28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:04:51.091890 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e1026e84-5efd-4e16-bccd-8f5e17a014ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:51.093834 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e1026e84-5efd-4e16-bccd-8f5e17a014ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:51.099077 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 677/1356] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:04:50 2022] DELETE /identity/v3/projects/48d87e6406724af3b1330e6f3cc15a28 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:04:51.104149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a8059-7d8d-4cd8-9b92-aa8b1c396254 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:51.104676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a8059-7d8d-4cd8-9b92-aa8b1c396254 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:51.104676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a8059-7d8d-4cd8-9b92-aa8b1c396254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:51.162554 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-b19a8059-7d8d-4cd8-9b92-aa8b1c396254 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:51.171630 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b19a8059-7d8d-4cd8-9b92-aa8b1c396254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:51.414441 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 680/1357] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:51.425697 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ccdfd6-92c2-41df-ac3f-054a1d71bcec None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:51.426508 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ccdfd6-92c2-41df-ac3f-054a1d71bcec None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:51.427180 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ccdfd6-92c2-41df-ac3f-054a1d71bcec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:51.491371 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-46ccdfd6-92c2-41df-ac3f-054a1d71bcec None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:51.499676 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-46ccdfd6-92c2-41df-ac3f-054a1d71bcec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:51.541166 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 678/1358] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:51.545329 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a8059-7d8d-4cd8-9b92-aa8b1c396254 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:51.566246 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a8059-7d8d-4cd8-9b92-aa8b1c396254 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:51.580548 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6cdf8a-27df-4ec8-9907-e39e67c5f563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:51.582320 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cdf8a-27df-4ec8-9907-e39e67c5f563 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:51.583191 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cdf8a-27df-4ec8-9907-e39e67c5f563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:51.583699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cdf8a-27df-4ec8-9907-e39e67c5f563 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:51.585947 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6cdf8a-27df-4ec8-9907-e39e67c5f563 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:04:51.586303 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6cdf8a-27df-4ec8-9907-e39e67c5f563 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:51.589870 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6cdf8a-27df-4ec8-9907-e39e67c5f563 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:51.648898 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ccdfd6-92c2-41df-ac3f-054a1d71bcec None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:51.651857 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-46ccdfd6-92c2-41df-ac3f-054a1d71bcec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:51.697012 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 681/1359] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:04:51.700126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6cdf8a-27df-4ec8-9907-e39e67c5f563 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:51.703043 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ccdfd6-92c2-41df-ac3f-054a1d71bcec None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:51.729201 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d878394-1415-4025-bd49-863992f0d981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:51.731688 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d878394-1415-4025-bd49-863992f0d981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:51.731973 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d878394-1415-4025-bd49-863992f0d981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:51.733440 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6cdf8a-27df-4ec8-9907-e39e67c5f563 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:51.733981 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d878394-1415-4025-bd49-863992f0d981 admin admin] PATH_INFO: `/v3/projects/7f038a048ecc436d857d0cb2866f5c59` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:51.738002 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d878394-1415-4025-bd49-863992f0d981 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f038a048ecc436d857d0cb2866f5c59)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:51.744185 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d878394-1415-4025-bd49-863992f0d981 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:51.751954 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851c9e4-77fc-4e32-bd46-fed60192cbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:51.752799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851c9e4-77fc-4e32-bd46-fed60192cbad admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:51.753103 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851c9e4-77fc-4e32-bd46-fed60192cbad admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:51.753383 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851c9e4-77fc-4e32-bd46-fed60192cbad admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:51.754020 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c851c9e4-77fc-4e32-bd46-fed60192cbad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:51.761375 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c851c9e4-77fc-4e32-bd46-fed60192cbad admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:51.890790 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 682/1360] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:04:51 2022] POST /identity/v3/projects => generated 401 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:51.898707 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6d878394-1415-4025-bd49-863992f0d981 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f038a048ecc436d857d0cb2866f5c59', 'request_id': 'req-6d878394-1415-4025-bd49-863992f0d981'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:51.899377 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a485bcb-eb34-49a0-9096-e5f10e3324f9 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:51.904128 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a485bcb-eb34-49a0-9096-e5f10e3324f9 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:51.904731 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a485bcb-eb34-49a0-9096-e5f10e3324f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:51.917340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6d878394-1415-4025-bd49-863992f0d981 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f038a048ecc436d857d0cb2866f5c59', 'request_id': 'req-6d878394-1415-4025-bd49-863992f0d981'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:51.929015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6d878394-1415-4025-bd49-863992f0d981 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f038a048ecc436d857d0cb2866f5c59', 'request_id': 'req-6d878394-1415-4025-bd49-863992f0d981'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:51.954669 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6d878394-1415-4025-bd49-863992f0d981 admin admin] The token cache is being invalidate because project 7f038a048ecc436d857d0cb2866f5c59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:04:51.955156 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6d878394-1415-4025-bd49-863992f0d981 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:51.955905 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6d878394-1415-4025-bd49-863992f0d981 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:51.958429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 679/1361] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:04:51 2022] DELETE /identity/v3/projects/7f038a048ecc436d857d0cb2866f5c59 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:04:51.959764 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d878394-1415-4025-bd49-863992f0d981 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:51.962588 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6d878394-1415-4025-bd49-863992f0d981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:51.992339 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-8a485bcb-eb34-49a0-9096-e5f10e3324f9 None None] MFA Rules not processed for user `9072ce66e7174837aeb8c7898026fcce`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:51.995853 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8a485bcb-eb34-49a0-9096-e5f10e3324f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:52.179101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 683/1362] 10.222.0.18 () {58 vars in 1015 bytes} [Thu Oct 20 06:04:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:52.184968 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a485bcb-eb34-49a0-9096-e5f10e3324f9 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:52.205455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8a485bcb-eb34-49a0-9096-e5f10e3324f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:52.229483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:52.265443 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:52.342673 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d878394-1415-4025-bd49-863992f0d981 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:52.426772 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a485bcb-eb34-49a0-9096-e5f10e3324f9 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:52.446219 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4cd639-aff7-4208-b34a-b8dc134487b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:52.447275 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4cd639-aff7-4208-b34a-b8dc134487b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:52.447535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4cd639-aff7-4208-b34a-b8dc134487b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:52.447907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4cd639-aff7-4208-b34a-b8dc134487b7 admin admin] PATH_INFO: `/v3/users/b4acf4f64a67401a8fc1336bd59b592f` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:52.449572 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4cd639-aff7-4208-b34a-b8dc134487b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4acf4f64a67401a8fc1336bd59b592f)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:52.458841 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4cd639-aff7-4208-b34a-b8dc134487b7 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:52.588895 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2e4cd639-aff7-4208-b34a-b8dc134487b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4acf4f64a67401a8fc1336bd59b592f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:52.589630 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2e4cd639-aff7-4208-b34a-b8dc134487b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4acf4f64a67401a8fc1336bd59b592f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:52.612002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2e4cd639-aff7-4208-b34a-b8dc134487b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4acf4f64a67401a8fc1336bd59b592f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:52.636794 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab6a51-6377-48d7-a7b6-8340aaa67ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:52.637385 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab6a51-6377-48d7-a7b6-8340aaa67ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:52.637811 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab6a51-6377-48d7-a7b6-8340aaa67ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:52.638008 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab6a51-6377-48d7-a7b6-8340aaa67ea2 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:52.639071 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bab6a51-6377-48d7-a7b6-8340aaa67ea2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:52.646035 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bab6a51-6377-48d7-a7b6-8340aaa67ea2 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:52.661928 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2e4cd639-aff7-4208-b34a-b8dc134487b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4acf4f64a67401a8fc1336bd59b592f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:52.665005 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2e4cd639-aff7-4208-b34a-b8dc134487b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4acf4f64a67401a8fc1336bd59b592f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:52.695938 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 684/1363] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:04:52 2022] DELETE /identity/v3/users/b4acf4f64a67401a8fc1336bd59b592f => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:04:52.697213 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4cd639-aff7-4208-b34a-b8dc134487b7 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:52.704205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 680/1364] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:04:51 2022] POST /identity/v3/users => generated 301 bytes in 745 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:52.704752 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2e4cd639-aff7-4208-b34a-b8dc134487b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:52.709721 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab6a51-6377-48d7-a7b6-8340aaa67ea2 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:52.713223 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1bab6a51-6377-48d7-a7b6-8340aaa67ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:52.747313 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab6a51-6377-48d7-a7b6-8340aaa67ea2 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:52.919709 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f44113-0e36-487a-93a6-3f6a7f46b151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:52.920316 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f44113-0e36-487a-93a6-3f6a7f46b151 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:52.920595 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f44113-0e36-487a-93a6-3f6a7f46b151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:52.920906 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f44113-0e36-487a-93a6-3f6a7f46b151 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:52.921562 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f44113-0e36-487a-93a6-3f6a7f46b151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:52.923219 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f44113-0e36-487a-93a6-3f6a7f46b151 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:52.941717 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4cd639-aff7-4208-b34a-b8dc134487b7 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:52.953246 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7083fb-9ec7-4fb6-8686-1825d5f22fd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:52.953895 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7083fb-9ec7-4fb6-8686-1825d5f22fd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:52.954190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7083fb-9ec7-4fb6-8686-1825d5f22fd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:52.954592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7083fb-9ec7-4fb6-8686-1825d5f22fd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:52.960572 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ca7083fb-9ec7-4fb6-8686-1825d5f22fd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:52.981243 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 681/1365] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:52.985921 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f44113-0e36-487a-93a6-3f6a7f46b151 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:52.987543 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7083fb-9ec7-4fb6-8686-1825d5f22fd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:52.989658 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7083fb-9ec7-4fb6-8686-1825d5f22fd5 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:53.006984 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 685/1366] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:04:53.009239 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f44113-0e36-487a-93a6-3f6a7f46b151 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:53.018308 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b9f3c-6c3b-4d68-8c4a-dfc4017ed9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:53.019765 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b9f3c-6c3b-4d68-8c4a-dfc4017ed9f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:53.020197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b9f3c-6c3b-4d68-8c4a-dfc4017ed9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:53.020711 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b9f3c-6c3b-4d68-8c4a-dfc4017ed9f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:53.021601 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b9f3c-6c3b-4d68-8c4a-dfc4017ed9f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:53.023346 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b9f3c-6c3b-4d68-8c4a-dfc4017ed9f3 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:53.045081 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 682/1367] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:53.048551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7083fb-9ec7-4fb6-8686-1825d5f22fd5 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:53.067676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7083fb-9ec7-4fb6-8686-1825d5f22fd5 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:53.083847 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812b574-1b70-4d50-96b1-f92b3283e4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:53.084544 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812b574-1b70-4d50-96b1-f92b3283e4bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:53.084847 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812b574-1b70-4d50-96b1-f92b3283e4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:53.087946 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812b574-1b70-4d50-96b1-f92b3283e4bc admin admin] PATH_INFO: `/v3/projects/09732b03e24c43a4aaaf4e80fa15dc94/users/cf78f545cf7943a6a24dbfcfd1555443/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:53.127433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4812b574-1b70-4d50-96b1-f92b3283e4bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=09732b03e24c43a4aaaf4e80fa15dc94, user_id=cf78f545cf7943a6a24dbfcfd1555443, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:53.129881 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4812b574-1b70-4d50-96b1-f92b3283e4bc admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:53.168085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 686/1368] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:04:53 2022] PUT /identity/v3/projects/09732b03e24c43a4aaaf4e80fa15dc94/users/cf78f545cf7943a6a24dbfcfd1555443/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:04:53.181252 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801c06e-fc61-4f16-b7cd-e43fdd35c4d3 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:53.181514 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801c06e-fc61-4f16-b7cd-e43fdd35c4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:53.182018 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801c06e-fc61-4f16-b7cd-e43fdd35c4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:53.245273 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3801c06e-fc61-4f16-b7cd-e43fdd35c4d3 None None] MFA Rules not processed for user `cf78f545cf7943a6a24dbfcfd1555443`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:53.246732 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3801c06e-fc61-4f16-b7cd-e43fdd35c4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:53.439326 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 683/1369] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:53 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:53.441935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812b574-1b70-4d50-96b1-f92b3283e4bc admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:53.447877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:53.514013 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46a9bf-70f3-4763-b00d-7b21422a1230 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:53.514301 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46a9bf-70f3-4763-b00d-7b21422a1230 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:53.514580 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46a9bf-70f3-4763-b00d-7b21422a1230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:53.529230 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:53.569374 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-9d46a9bf-70f3-4763-b00d-7b21422a1230 None None] MFA Rules not processed for user `6da8a7cb6ef849faad7d7091a1095ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:53.577456 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-9d46a9bf-70f3-4763-b00d-7b21422a1230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:53.664351 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812b574-1b70-4d50-96b1-f92b3283e4bc admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:53.682336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b2b54-7bb3-4e70-87f7-2dbef5cb0992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:53.683070 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b2b54-7bb3-4e70-87f7-2dbef5cb0992 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:53.683405 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b2b54-7bb3-4e70-87f7-2dbef5cb0992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:53.683679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b2b54-7bb3-4e70-87f7-2dbef5cb0992 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:53.684371 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b2b54-7bb3-4e70-87f7-2dbef5cb0992 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:53.686404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b2b54-7bb3-4e70-87f7-2dbef5cb0992 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:53.803631 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 687/1370] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:04:53 2022] POST /identity/v3/projects => generated 401 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:53.805167 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b2b54-7bb3-4e70-87f7-2dbef5cb0992 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:53.833280 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 684/1371] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:54.102179 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b2b54-7bb3-4e70-87f7-2dbef5cb0992 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:54.121815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914f5b6-a4eb-40a2-a149-dc3ba4d4cd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:54.122755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914f5b6-a4eb-40a2-a149-dc3ba4d4cd52 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:54.123240 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914f5b6-a4eb-40a2-a149-dc3ba4d4cd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:54.123829 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914f5b6-a4eb-40a2-a149-dc3ba4d4cd52 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:54.124719 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b914f5b6-a4eb-40a2-a149-dc3ba4d4cd52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:54.131658 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b914f5b6-a4eb-40a2-a149-dc3ba4d4cd52 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:54.184756 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 688/1372] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:04:53 2022] POST /identity/v3/users => generated 307 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:54.188956 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46a9bf-70f3-4763-b00d-7b21422a1230 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:54.213693 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46a9bf-70f3-4763-b00d-7b21422a1230 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:54.230022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdfebf7-b904-4149-b12b-4ecdc3893e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:54.230728 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfebf7-b904-4149-b12b-4ecdc3893e6c admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:54.231006 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfebf7-b904-4149-b12b-4ecdc3893e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:54.231943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfebf7-b904-4149-b12b-4ecdc3893e6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:54.233523 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdfebf7-b904-4149-b12b-4ecdc3893e6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:54.237668 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdfebf7-b904-4149-b12b-4ecdc3893e6c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:54.262225 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 685/1373] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:54.262426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914f5b6-a4eb-40a2-a149-dc3ba4d4cd52 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:54.297566 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914f5b6-a4eb-40a2-a149-dc3ba4d4cd52 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:54.336482 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43e1c6-d096-4ef2-b84d-4db70c2eff06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:54.337958 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43e1c6-d096-4ef2-b84d-4db70c2eff06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:54.338501 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43e1c6-d096-4ef2-b84d-4db70c2eff06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:54.339011 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43e1c6-d096-4ef2-b84d-4db70c2eff06 admin admin] PATH_INFO: `/v3/projects/43cbf0d29a0243d498a7ed1acac305f3/users/5a99d9ac88f54f989b2ff9d7496cf583/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:54.366172 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f43e1c6-d096-4ef2-b84d-4db70c2eff06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43cbf0d29a0243d498a7ed1acac305f3, user_id=5a99d9ac88f54f989b2ff9d7496cf583, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:54.372138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f43e1c6-d096-4ef2-b84d-4db70c2eff06 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:54.432235 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdfebf7-b904-4149-b12b-4ecdc3893e6c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:54.440196 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 689/1374] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:04:54 2022] PUT /identity/v3/projects/43cbf0d29a0243d498a7ed1acac305f3/users/5a99d9ac88f54f989b2ff9d7496cf583/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:04:54.448333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdfebf7-b904-4149-b12b-4ecdc3893e6c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:54.602994 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd97ae-e97b-491b-82b8-dcf43c56e062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:54.604035 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd97ae-e97b-491b-82b8-dcf43c56e062 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:54.605701 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd97ae-e97b-491b-82b8-dcf43c56e062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:54.606958 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd97ae-e97b-491b-82b8-dcf43c56e062 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:54.609026 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebd97ae-e97b-491b-82b8-dcf43c56e062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:54.611902 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebd97ae-e97b-491b-82b8-dcf43c56e062 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:54.637306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 686/1375] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:54.641745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43e1c6-d096-4ef2-b84d-4db70c2eff06 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:54.667440 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43e1c6-d096-4ef2-b84d-4db70c2eff06 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:54.677609 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1492fbc-364f-4f3a-a842-1faa912386e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:54.678907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1492fbc-364f-4f3a-a842-1faa912386e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:54.679251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1492fbc-364f-4f3a-a842-1faa912386e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:54.680620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1492fbc-364f-4f3a-a842-1faa912386e1 admin admin] PATH_INFO: `/v3/projects/43cbf0d29a0243d498a7ed1acac305f3/users/5a99d9ac88f54f989b2ff9d7496cf583/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:54.685853 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1492fbc-364f-4f3a-a842-1faa912386e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43cbf0d29a0243d498a7ed1acac305f3, user_id=5a99d9ac88f54f989b2ff9d7496cf583, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:54.724109 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1492fbc-364f-4f3a-a842-1faa912386e1 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:54.745130 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 690/1376] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:04:54 2022] PUT /identity/v3/projects/43cbf0d29a0243d498a7ed1acac305f3/users/5a99d9ac88f54f989b2ff9d7496cf583/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:04:54.762822 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877e8c4-b5ae-4043-8b85-c7ccff8fa0d7 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:54.763401 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877e8c4-b5ae-4043-8b85-c7ccff8fa0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:54.763824 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877e8c4-b5ae-4043-8b85-c7ccff8fa0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:54.851280 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-8877e8c4-b5ae-4043-8b85-c7ccff8fa0d7 None None] MFA Rules not processed for user `5a99d9ac88f54f989b2ff9d7496cf583`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:54.853729 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8877e8c4-b5ae-4043-8b85-c7ccff8fa0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:55.063129 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 687/1377] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:54 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:55.075479 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1492fbc-364f-4f3a-a842-1faa912386e1 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:55.077729 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b1492fbc-364f-4f3a-a842-1faa912386e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:55.251373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:55.305985 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1492fbc-364f-4f3a-a842-1faa912386e1 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:55.334709 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43df43-b5aa-49c7-8198-260448e42364 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:55.335498 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43df43-b5aa-49c7-8198-260448e42364 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:55.336516 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43df43-b5aa-49c7-8198-260448e42364 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:55.341467 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43df43-b5aa-49c7-8198-260448e42364 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:55.387087 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43df43-b5aa-49c7-8198-260448e42364 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:55.396083 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43df43-b5aa-49c7-8198-260448e42364 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:55.447207 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 691/1378] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:55 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:04:55.585042 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877e8c4-b5ae-4043-8b85-c7ccff8fa0d7 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:55.606117 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877e8c4-b5ae-4043-8b85-c7ccff8fa0d7 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:55.619262 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8f940-c79e-4a76-ab08-3f4dc035846d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:55.620264 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8f940-c79e-4a76-ab08-3f4dc035846d service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:55.620568 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8f940-c79e-4a76-ab08-3f4dc035846d service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:55.622109 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8f940-c79e-4a76-ab08-3f4dc035846d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:55.643729 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:55.737139 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8f940-c79e-4a76-ab08-3f4dc035846d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:55.738940 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8f940-c79e-4a76-ab08-3f4dc035846d service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:55.759313 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 688/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:04:55 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:04:56.718087 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43df43-b5aa-49c7-8198-260448e42364 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:56.719599 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-fa43df43-b5aa-49c7-8198-260448e42364 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:57.131935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43df43-b5aa-49c7-8198-260448e42364 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:57.144483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afdbc79-abb4-4960-ba2d-dd0eb820b48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:57.148422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afdbc79-abb4-4960-ba2d-dd0eb820b48f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:57.149358 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afdbc79-abb4-4960-ba2d-dd0eb820b48f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:57.149998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afdbc79-abb4-4960-ba2d-dd0eb820b48f admin admin] PATH_INFO: `/v3/projects/4cf38fcec6b140d8adbbb94ba4d2fc1a` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:57.155068 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afdbc79-abb4-4960-ba2d-dd0eb820b48f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cf38fcec6b140d8adbbb94ba4d2fc1a)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:57.159845 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afdbc79-abb4-4960-ba2d-dd0eb820b48f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:57.164173 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b00993-2f70-4323-801a-aed629f9568f None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:57.166424 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b00993-2f70-4323-801a-aed629f9568f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:57.166592 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b00993-2f70-4323-801a-aed629f9568f None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:57.175479 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 689/1380] 10.222.0.20 () {54 vars in 921 bytes} [Thu Oct 20 06:04:57 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 06:04:57.280991 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6afdbc79-abb4-4960-ba2d-dd0eb820b48f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cf38fcec6b140d8adbbb94ba4d2fc1a', 'request_id': 'req-6afdbc79-abb4-4960-ba2d-dd0eb820b48f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:57.320792 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6afdbc79-abb4-4960-ba2d-dd0eb820b48f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cf38fcec6b140d8adbbb94ba4d2fc1a', 'request_id': 'req-6afdbc79-abb4-4960-ba2d-dd0eb820b48f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:57.348300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6afdbc79-abb4-4960-ba2d-dd0eb820b48f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cf38fcec6b140d8adbbb94ba4d2fc1a', 'request_id': 'req-6afdbc79-abb4-4960-ba2d-dd0eb820b48f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:57.411224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6afdbc79-abb4-4960-ba2d-dd0eb820b48f admin admin] The token cache is being invalidate because project 4cf38fcec6b140d8adbbb94ba4d2fc1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:04:57.411416 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6afdbc79-abb4-4960-ba2d-dd0eb820b48f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:57.412107 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6afdbc79-abb4-4960-ba2d-dd0eb820b48f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:04:57.421425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 692/1381] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:04:56 2022] DELETE /identity/v3/projects/4cf38fcec6b140d8adbbb94ba4d2fc1a => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:04:57.427739 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31872bb-3679-4932-9091-f5bbb37a3fe4 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:57.428024 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31872bb-3679-4932-9091-f5bbb37a3fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:57.428301 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31872bb-3679-4932-9091-f5bbb37a3fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:57.451799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147aaa8f-6b08-4b7a-bf73-8b1ccd0fdb00 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:57.452565 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147aaa8f-6b08-4b7a-bf73-8b1ccd0fdb00 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:57.452928 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147aaa8f-6b08-4b7a-bf73-8b1ccd0fdb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:57.526571 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-b31872bb-3679-4932-9091-f5bbb37a3fe4 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:57.527841 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b31872bb-3679-4932-9091-f5bbb37a3fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:57.545396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-147aaa8f-6b08-4b7a-bf73-8b1ccd0fdb00 None None] MFA Rules not processed for user `ae2c1c886cd94d9f90082f66e7e8c9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:57.554498 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-147aaa8f-6b08-4b7a-bf73-8b1ccd0fdb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:57.788552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 693/1382] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Oct 20 06:04:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:57.840759 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 690/1383] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:57.849633 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005a49b-33a4-49d1-8eb8-35d409e07ecb None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:57.850355 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005a49b-33a4-49d1-8eb8-35d409e07ecb None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:57.850866 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005a49b-33a4-49d1-8eb8-35d409e07ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:57.865683 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:57.893819 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-6005a49b-33a4-49d1-8eb8-35d409e07ecb None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:57.897635 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6005a49b-33a4-49d1-8eb8-35d409e07ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:57.936624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 694/1384] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:57.937918 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31872bb-3679-4932-9091-f5bbb37a3fe4 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:57.963032 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31872bb-3679-4932-9091-f5bbb37a3fe4 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:57.998518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a59c4-8026-493e-a4b2-68678f3eb6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:58.005790 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a59c4-8026-493e-a4b2-68678f3eb6d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:58.006344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a59c4-8026-493e-a4b2-68678f3eb6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:58.008367 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a59c4-8026-493e-a4b2-68678f3eb6d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:58.009348 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a59c4-8026-493e-a4b2-68678f3eb6d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:04:58.010888 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a59c4-8026-493e-a4b2-68678f3eb6d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:58.013273 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a59c4-8026-493e-a4b2-68678f3eb6d7 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:58.084472 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 691/1385] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:04:58.086016 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005a49b-33a4-49d1-8eb8-35d409e07ecb None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:58.102323 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005a49b-33a4-49d1-8eb8-35d409e07ecb None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:58.121737 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25e4ea-3fd8-42cc-af8e-6f32633284a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:58.122542 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25e4ea-3fd8-42cc-af8e-6f32633284a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:58.122941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25e4ea-3fd8-42cc-af8e-6f32633284a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:58.124235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25e4ea-3fd8-42cc-af8e-6f32633284a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:58.126994 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c25e4ea-3fd8-42cc-af8e-6f32633284a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:58.134559 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c25e4ea-3fd8-42cc-af8e-6f32633284a7 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:58.265521 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a59c4-8026-493e-a4b2-68678f3eb6d7 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:58.266672 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 695/1386] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:04:58 2022] POST /identity/v3/projects => generated 387 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:58.284573 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a59c4-8026-493e-a4b2-68678f3eb6d7 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:58.586445 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6987328-ff7f-469c-bf7a-0bd4ec58b5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:58.587157 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6987328-ff7f-469c-bf7a-0bd4ec58b5d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:58.587389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6987328-ff7f-469c-bf7a-0bd4ec58b5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:58.587774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6987328-ff7f-469c-bf7a-0bd4ec58b5d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:58.588350 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6987328-ff7f-469c-bf7a-0bd4ec58b5d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:58.590776 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6987328-ff7f-469c-bf7a-0bd4ec58b5d3 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:58.750160 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 692/1387] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:04:58 2022] POST /identity/v3/users => generated 294 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:58.754513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25e4ea-3fd8-42cc-af8e-6f32633284a7 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:58.771921 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25e4ea-3fd8-42cc-af8e-6f32633284a7 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:58.790702 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28082d52-bc67-4606-b054-86f3392666e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:58.797126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28082d52-bc67-4606-b054-86f3392666e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:58.797680 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28082d52-bc67-4606-b054-86f3392666e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:58.800254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28082d52-bc67-4606-b054-86f3392666e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:58.803060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28082d52-bc67-4606-b054-86f3392666e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:58.805528 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28082d52-bc67-4606-b054-86f3392666e0 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:58.845888 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6987328-ff7f-469c-bf7a-0bd4ec58b5d3 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:58.846108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 696/1388] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:58.874395 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6987328-ff7f-469c-bf7a-0bd4ec58b5d3 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:58.897356 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17213afd-1fcc-4e51-97bc-9b7310616cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:58.898239 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17213afd-1fcc-4e51-97bc-9b7310616cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:58.898609 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17213afd-1fcc-4e51-97bc-9b7310616cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:58.898891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17213afd-1fcc-4e51-97bc-9b7310616cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:58.899551 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17213afd-1fcc-4e51-97bc-9b7310616cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:58.905614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17213afd-1fcc-4e51-97bc-9b7310616cc9 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:58.944012 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28082d52-bc67-4606-b054-86f3392666e0 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:58.946725 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 693/1389] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:04:58.970775 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28082d52-bc67-4606-b054-86f3392666e0 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:58.994022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef966a-eed7-42e0-86b3-9e910a0e6fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:58.994912 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef966a-eed7-42e0-86b3-9e910a0e6fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:58.995301 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef966a-eed7-42e0-86b3-9e910a0e6fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:58.995689 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef966a-eed7-42e0-86b3-9e910a0e6fc2 admin admin] PATH_INFO: `/v3/projects/235d4af7238b4e809989de633cc3cf0e/users/df4af7e4fc1b4a6b9fa37accefb0e00d/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:59.041016 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef966a-eed7-42e0-86b3-9e910a0e6fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=235d4af7238b4e809989de633cc3cf0e, user_id=df4af7e4fc1b4a6b9fa37accefb0e00d, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:59.044259 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef966a-eed7-42e0-86b3-9e910a0e6fc2 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:59.080094 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 697/1390] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:04:58 2022] PUT /identity/v3/projects/235d4af7238b4e809989de633cc3cf0e/users/df4af7e4fc1b4a6b9fa37accefb0e00d/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:04:59.091961 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f56f8-6b05-417c-8b4d-013e5477300a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:59.092244 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f56f8-6b05-417c-8b4d-013e5477300a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:59.092521 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f56f8-6b05-417c-8b4d-013e5477300a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:59.199728 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-a50f56f8-6b05-417c-8b4d-013e5477300a None None] MFA Rules not processed for user `df4af7e4fc1b4a6b9fa37accefb0e00d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:04:59.201029 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a50f56f8-6b05-417c-8b4d-013e5477300a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:04:59.243945 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:04:59.408402 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef966a-eed7-42e0-86b3-9e910a0e6fc2 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:59.411628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 694/1391] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:04:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:04:59.615732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef966a-eed7-42e0-86b3-9e910a0e6fc2 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:59.643099 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e344b2f-fd07-4b7b-a8d1-5b4baa4ca8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:59.644226 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e344b2f-fd07-4b7b-a8d1-5b4baa4ca8f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:59.644519 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e344b2f-fd07-4b7b-a8d1-5b4baa4ca8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:59.644978 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e344b2f-fd07-4b7b-a8d1-5b4baa4ca8f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:59.645530 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e344b2f-fd07-4b7b-a8d1-5b4baa4ca8f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:59.648819 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e344b2f-fd07-4b7b-a8d1-5b4baa4ca8f4 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:04:59.723675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 698/1392] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:04:59 2022] POST /identity/v3/projects => generated 389 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:04:59.727761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f56f8-6b05-417c-8b4d-013e5477300a None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:04:59.755508 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f56f8-6b05-417c-8b4d-013e5477300a None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:04:59.928387 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0d2e2-1745-4cb8-b52d-da8307e1cb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:04:59.931511 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0d2e2-1745-4cb8-b52d-da8307e1cb9c admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:04:59.932548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0d2e2-1745-4cb8-b52d-da8307e1cb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:04:59.933075 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0d2e2-1745-4cb8-b52d-da8307e1cb9c admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:04:59.953697 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0d2e2-1745-4cb8-b52d-da8307e1cb9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:04:59.963761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0d2e2-1745-4cb8-b52d-da8307e1cb9c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:00.044080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e344b2f-fd07-4b7b-a8d1-5b4baa4ca8f4 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:00.058418 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 695/1393] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:04:59 2022] POST /identity/v3/users => generated 301 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:00.068338 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e344b2f-fd07-4b7b-a8d1-5b4baa4ca8f4 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:00.084685 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa0c9f-348f-4fc9-853f-97f8cd4bb949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:00.089035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa0c9f-348f-4fc9-853f-97f8cd4bb949 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:00.090195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa0c9f-348f-4fc9-853f-97f8cd4bb949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:00.091135 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa0c9f-348f-4fc9-853f-97f8cd4bb949 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:00.093865 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa0c9f-348f-4fc9-853f-97f8cd4bb949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:00.097804 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa0c9f-348f-4fc9-853f-97f8cd4bb949 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:00.139714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0d2e2-1745-4cb8-b52d-da8307e1cb9c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:00.142233 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 699/1394] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:00.159388 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0d2e2-1745-4cb8-b52d-da8307e1cb9c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:00.175690 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84fc6d-93f6-45c7-bd82-2977acb0a7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:00.193086 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84fc6d-93f6-45c7-bd82-2977acb0a7ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:00.196155 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84fc6d-93f6-45c7-bd82-2977acb0a7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:00.198379 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84fc6d-93f6-45c7-bd82-2977acb0a7ef admin admin] PATH_INFO: `/v3/projects/f0b123a72e924d5ca375c5230667f099/users/51b40da71c6f4b909265a068ed33950c/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:00.251373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84fc6d-93f6-45c7-bd82-2977acb0a7ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0b123a72e924d5ca375c5230667f099, user_id=51b40da71c6f4b909265a068ed33950c, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:00.282469 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84fc6d-93f6-45c7-bd82-2977acb0a7ef admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:00.353102 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 696/1395] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:00 2022] PUT /identity/v3/projects/f0b123a72e924d5ca375c5230667f099/users/51b40da71c6f4b909265a068ed33950c/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:00.357776 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa0c9f-348f-4fc9-853f-97f8cd4bb949 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:00.379676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa0c9f-348f-4fc9-853f-97f8cd4bb949 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:00.473853 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84fc6d-93f6-45c7-bd82-2977acb0a7ef admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:00.476384 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2e84fc6d-93f6-45c7-bd82-2977acb0a7ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:00.512608 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8c65a-7fd8-4e55-b7ab-100893a42736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:00.517771 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8c65a-7fd8-4e55-b7ab-100893a42736 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:00.518103 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8c65a-7fd8-4e55-b7ab-100893a42736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:00.518378 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8c65a-7fd8-4e55-b7ab-100893a42736 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:00.521200 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e8c65a-7fd8-4e55-b7ab-100893a42736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:00.523775 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e8c65a-7fd8-4e55-b7ab-100893a42736 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:00.564736 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 700/1396] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:00.567676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8c65a-7fd8-4e55-b7ab-100893a42736 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:00.607358 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8c65a-7fd8-4e55-b7ab-100893a42736 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:00.624841 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6520b-5b4d-4822-92e2-6b38b2c5ad90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:00.626202 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6520b-5b4d-4822-92e2-6b38b2c5ad90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:00.626780 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6520b-5b4d-4822-92e2-6b38b2c5ad90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:00.627221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6520b-5b4d-4822-92e2-6b38b2c5ad90 admin admin] PATH_INFO: `/v3/projects/f0b123a72e924d5ca375c5230667f099/users/51b40da71c6f4b909265a068ed33950c/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:00.632265 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6520b-5b4d-4822-92e2-6b38b2c5ad90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0b123a72e924d5ca375c5230667f099, user_id=51b40da71c6f4b909265a068ed33950c, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:00.637199 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6520b-5b4d-4822-92e2-6b38b2c5ad90 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:00.647277 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:00.677700 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 701/1397] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:00 2022] PUT /identity/v3/projects/f0b123a72e924d5ca375c5230667f099/users/51b40da71c6f4b909265a068ed33950c/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:00.688816 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4511bc2-2136-4437-9906-6ea63155a27f None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:00.689094 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4511bc2-2136-4437-9906-6ea63155a27f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:00.689652 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4511bc2-2136-4437-9906-6ea63155a27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:00.739949 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84fc6d-93f6-45c7-bd82-2977acb0a7ef admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:00.766028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d4511bc2-2136-4437-9906-6ea63155a27f None None] MFA Rules not processed for user `51b40da71c6f4b909265a068ed33950c`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:00.782694 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d4511bc2-2136-4437-9906-6ea63155a27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:00.796147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:00.954727 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 702/1398] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:00 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:05:00.962550 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4511bc2-2136-4437-9906-6ea63155a27f None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:00.964336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d4511bc2-2136-4437-9906-6ea63155a27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:00.969963 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b821f22-507d-4510-b5a6-1a0e78baf167 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:00.970608 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b821f22-507d-4510-b5a6-1a0e78baf167 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:00.970894 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b821f22-507d-4510-b5a6-1a0e78baf167 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:00.972064 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b821f22-507d-4510-b5a6-1a0e78baf167 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:00.989063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b821f22-507d-4510-b5a6-1a0e78baf167 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:00.990840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b821f22-507d-4510-b5a6-1a0e78baf167 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:01.003780 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 697/1399] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:01.066948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4511bc2-2136-4437-9906-6ea63155a27f None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:01.073656 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244856a-aaf9-4e12-a1a8-ef5b8c4a2491 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:01.075587 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244856a-aaf9-4e12-a1a8-ef5b8c4a2491 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:01.076218 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244856a-aaf9-4e12-a1a8-ef5b8c4a2491 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:01.076630 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244856a-aaf9-4e12-a1a8-ef5b8c4a2491 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:01.094509 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244856a-aaf9-4e12-a1a8-ef5b8c4a2491 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:01.097063 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244856a-aaf9-4e12-a1a8-ef5b8c4a2491 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:01.116628 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 703/1400] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:00 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:01.363987 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b821f22-507d-4510-b5a6-1a0e78baf167 service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:01.512608 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b821f22-507d-4510-b5a6-1a0e78baf167 service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:01.527617 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832040f-35be-43b8-a11b-073db1b7d30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:01.528294 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832040f-35be-43b8-a11b-073db1b7d30e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:01.531675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832040f-35be-43b8-a11b-073db1b7d30e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:01.532025 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832040f-35be-43b8-a11b-073db1b7d30e admin admin] PATH_INFO: `/v3/users/df4af7e4fc1b4a6b9fa37accefb0e00d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:01.536554 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4832040f-35be-43b8-a11b-073db1b7d30e admin admin] RBAC: Authorizing `identity:delete_user(user_id=df4af7e4fc1b4a6b9fa37accefb0e00d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:01.540781 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4832040f-35be-43b8-a11b-073db1b7d30e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:01.679842 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4832040f-35be-43b8-a11b-073db1b7d30e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df4af7e4fc1b4a6b9fa37accefb0e00d'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:01.682518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4832040f-35be-43b8-a11b-073db1b7d30e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df4af7e4fc1b4a6b9fa37accefb0e00d'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:01.703446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4832040f-35be-43b8-a11b-073db1b7d30e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df4af7e4fc1b4a6b9fa37accefb0e00d'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:01.726781 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4832040f-35be-43b8-a11b-073db1b7d30e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df4af7e4fc1b4a6b9fa37accefb0e00d'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:01.744729 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4832040f-35be-43b8-a11b-073db1b7d30e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df4af7e4fc1b4a6b9fa37accefb0e00d'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:01.758444 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f13de-48c9-42b2-a40f-5d23af1b38a1 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:01.758837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f13de-48c9-42b2-a40f-5d23af1b38a1 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:01.759434 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f13de-48c9-42b2-a40f-5d23af1b38a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:01.773862 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 698/1401] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:01 2022] DELETE /identity/v3/users/df4af7e4fc1b4a6b9fa37accefb0e00d => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:01.792274 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a63f13de-48c9-42b2-a40f-5d23af1b38a1 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:01.793740 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a63f13de-48c9-42b2-a40f-5d23af1b38a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:01.893260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 704/1402] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:05:01.900095 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832040f-35be-43b8-a11b-073db1b7d30e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:01.901454 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e0047-c7f3-4985-b78e-dc7d0df68523 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:01.901909 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e0047-c7f3-4985-b78e-dc7d0df68523 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:01.902280 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e0047-c7f3-4985-b78e-dc7d0df68523 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:01.902403 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-4832040f-35be-43b8-a11b-073db1b7d30e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:01.904469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 705/1403] 10.222.0.20 () {54 vars in 921 bytes} [Thu Oct 20 06:05:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 06:05:01.919379 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfea88-620d-4bd8-8b29-1740819c3e18 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:01.920786 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfea88-620d-4bd8-8b29-1740819c3e18 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:01.921608 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfea88-620d-4bd8-8b29-1740819c3e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:01.935907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:01.994998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a1dfea88-620d-4bd8-8b29-1740819c3e18 None None] MFA Rules not processed for user `ae2c1c886cd94d9f90082f66e7e8c9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:01.996580 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a1dfea88-620d-4bd8-8b29-1740819c3e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:01.996763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832040f-35be-43b8-a11b-073db1b7d30e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:02.004667 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21178f-07e2-42ea-a87d-939ec9d387b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:02.006004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21178f-07e2-42ea-a87d-939ec9d387b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:02.006295 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21178f-07e2-42ea-a87d-939ec9d387b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:02.006567 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21178f-07e2-42ea-a87d-939ec9d387b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:02.021972 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b21178f-07e2-42ea-a87d-939ec9d387b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:02.023869 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b21178f-07e2-42ea-a87d-939ec9d387b1 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:02.049524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:02.073940 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 699/1404] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:02.156791 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 706/1405] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Oct 20 06:05:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:02.170497 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21178f-07e2-42ea-a87d-939ec9d387b1 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:02.172606 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-4b21178f-07e2-42ea-a87d-939ec9d387b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:02.196022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21178f-07e2-42ea-a87d-939ec9d387b1 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:02.202484 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f8168-499a-425f-8bd9-289bdc2c62eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:02.203176 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f8168-499a-425f-8bd9-289bdc2c62eb service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:02.203527 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f8168-499a-425f-8bd9-289bdc2c62eb service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:02.203868 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f8168-499a-425f-8bd9-289bdc2c62eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:02.218945 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f8168-499a-425f-8bd9-289bdc2c62eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:02.221113 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f8168-499a-425f-8bd9-289bdc2c62eb service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:02.255064 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 700/1406] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:02.469187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfea88-620d-4bd8-8b29-1740819c3e18 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:02.470679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a1dfea88-620d-4bd8-8b29-1740819c3e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:02.502356 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfea88-620d-4bd8-8b29-1740819c3e18 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:02.516745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae9867-9274-4561-bbf2-fe32714db29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:02.517420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae9867-9274-4561-bbf2-fe32714db29b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:02.517680 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae9867-9274-4561-bbf2-fe32714db29b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:02.517950 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae9867-9274-4561-bbf2-fe32714db29b admin admin] PATH_INFO: `/v3/projects/235d4af7238b4e809989de633cc3cf0e` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:02.519217 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae9867-9274-4561-bbf2-fe32714db29b admin admin] RBAC: Authorizing `identity:delete_project(project_id=235d4af7238b4e809989de633cc3cf0e)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:02.522224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae9867-9274-4561-bbf2-fe32714db29b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:02.576202 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ffae9867-9274-4561-bbf2-fe32714db29b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '235d4af7238b4e809989de633cc3cf0e', 'request_id': 'req-ffae9867-9274-4561-bbf2-fe32714db29b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:02.584306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ffae9867-9274-4561-bbf2-fe32714db29b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '235d4af7238b4e809989de633cc3cf0e', 'request_id': 'req-ffae9867-9274-4561-bbf2-fe32714db29b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:02.591716 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ffae9867-9274-4561-bbf2-fe32714db29b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '235d4af7238b4e809989de633cc3cf0e', 'request_id': 'req-ffae9867-9274-4561-bbf2-fe32714db29b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:02.607275 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ffae9867-9274-4561-bbf2-fe32714db29b admin admin] The token cache is being invalidate because project 235d4af7238b4e809989de633cc3cf0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:02.608067 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ffae9867-9274-4561-bbf2-fe32714db29b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:02.608182 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ffae9867-9274-4561-bbf2-fe32714db29b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:02.610033 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 707/1407] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:02 2022] DELETE /identity/v3/projects/235d4af7238b4e809989de633cc3cf0e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:02.614885 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f8168-499a-425f-8bd9-289bdc2c62eb service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:02.632384 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f8168-499a-425f-8bd9-289bdc2c62eb service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:02.693848 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c37817-f934-4496-afca-8b1845de56b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:02.694498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c37817-f934-4496-afca-8b1845de56b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:02.694778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c37817-f934-4496-afca-8b1845de56b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:02.695051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c37817-f934-4496-afca-8b1845de56b1 admin admin] PATH_INFO: `/v3/users/51b40da71c6f4b909265a068ed33950c` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:02.696979 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c37817-f934-4496-afca-8b1845de56b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51b40da71c6f4b909265a068ed33950c)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:02.698915 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c37817-f934-4496-afca-8b1845de56b1 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:02.753159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c8c37817-f934-4496-afca-8b1845de56b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51b40da71c6f4b909265a068ed33950c'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:02.754653 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c8c37817-f934-4496-afca-8b1845de56b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51b40da71c6f4b909265a068ed33950c'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:02.763181 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c8c37817-f934-4496-afca-8b1845de56b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51b40da71c6f4b909265a068ed33950c'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:02.779064 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c8c37817-f934-4496-afca-8b1845de56b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51b40da71c6f4b909265a068ed33950c'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:02.786388 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c8c37817-f934-4496-afca-8b1845de56b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51b40da71c6f4b909265a068ed33950c'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:02.792749 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 701/1408] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:02 2022] DELETE /identity/v3/users/51b40da71c6f4b909265a068ed33950c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:03.284968 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae9867-9274-4561-bbf2-fe32714db29b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:03.287366 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ffae9867-9274-4561-bbf2-fe32714db29b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:03.307978 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae9867-9274-4561-bbf2-fe32714db29b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:03.390879 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df8517-aa5c-44ea-92e7-1b6989b824a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:03.391671 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df8517-aa5c-44ea-92e7-1b6989b824a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:03.391958 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df8517-aa5c-44ea-92e7-1b6989b824a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:03.392228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df8517-aa5c-44ea-92e7-1b6989b824a9 admin admin] PATH_INFO: `/v3/projects/f0b123a72e924d5ca375c5230667f099` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:03.393552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df8517-aa5c-44ea-92e7-1b6989b824a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0b123a72e924d5ca375c5230667f099)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:03.395947 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df8517-aa5c-44ea-92e7-1b6989b824a9 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:03.465068 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:03.478872 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-31df8517-aa5c-44ea-92e7-1b6989b824a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0b123a72e924d5ca375c5230667f099', 'request_id': 'req-31df8517-aa5c-44ea-92e7-1b6989b824a9'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:03.509137 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-31df8517-aa5c-44ea-92e7-1b6989b824a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0b123a72e924d5ca375c5230667f099', 'request_id': 'req-31df8517-aa5c-44ea-92e7-1b6989b824a9'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:03.532302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-31df8517-aa5c-44ea-92e7-1b6989b824a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0b123a72e924d5ca375c5230667f099', 'request_id': 'req-31df8517-aa5c-44ea-92e7-1b6989b824a9'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:03.557539 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-31df8517-aa5c-44ea-92e7-1b6989b824a9 admin admin] The token cache is being invalidate because project f0b123a72e924d5ca375c5230667f099 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:03.558222 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-31df8517-aa5c-44ea-92e7-1b6989b824a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:03.562906 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-31df8517-aa5c-44ea-92e7-1b6989b824a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:03.566531 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 708/1409] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:03 2022] DELETE /identity/v3/projects/f0b123a72e924d5ca375c5230667f099 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:03.572132 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c544cd3-6b31-4878-889b-a3de091305c8 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:03.572380 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c544cd3-6b31-4878-889b-a3de091305c8 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:03.573132 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c544cd3-6b31-4878-889b-a3de091305c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:03.607291 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-6c544cd3-6b31-4878-889b-a3de091305c8 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:03.608904 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6c544cd3-6b31-4878-889b-a3de091305c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:03.658422 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:03.732235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d3ab5-9280-44ac-bd3b-427fa727fadf None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:03.732542 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d3ab5-9280-44ac-bd3b-427fa727fadf None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:03.732835 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d3ab5-9280-44ac-bd3b-427fa727fadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:03.747380 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 702/1410] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:05:03.786553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-109d3ab5-9280-44ac-bd3b-427fa727fadf None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:03.794278 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-109d3ab5-9280-44ac-bd3b-427fa727fadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:03.814598 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 709/1411] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:03.819336 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c544cd3-6b31-4878-889b-a3de091305c8 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:03.831448 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c544cd3-6b31-4878-889b-a3de091305c8 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:03.842245 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebba435-9821-4d7b-bb94-ae77f86169df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:03.842916 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebba435-9821-4d7b-bb94-ae77f86169df admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:03.843181 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebba435-9821-4d7b-bb94-ae77f86169df admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:03.844909 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebba435-9821-4d7b-bb94-ae77f86169df admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:03.845592 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebba435-9821-4d7b-bb94-ae77f86169df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:05:03.853365 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebba435-9821-4d7b-bb94-ae77f86169df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:03.855641 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebba435-9821-4d7b-bb94-ae77f86169df admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:03.895629 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 703/1412] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:05:03.899995 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d3ab5-9280-44ac-bd3b-427fa727fadf None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:03.912827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d3ab5-9280-44ac-bd3b-427fa727fadf None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:03.917719 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fcce48-30a1-49ec-8719-228e8994052b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:03.918582 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcce48-30a1-49ec-8719-228e8994052b admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:03.918931 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcce48-30a1-49ec-8719-228e8994052b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:03.919274 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcce48-30a1-49ec-8719-228e8994052b admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:03.919978 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fcce48-30a1-49ec-8719-228e8994052b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:03.922301 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fcce48-30a1-49ec-8719-228e8994052b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:03.995539 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebba435-9821-4d7b-bb94-ae77f86169df admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:04.002964 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 710/1413] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:03 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:04.007809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebba435-9821-4d7b-bb94-ae77f86169df admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:04.148764 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb611f-c943-4580-b7ec-8a987962bb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:04.149404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb611f-c943-4580-b7ec-8a987962bb18 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:04.149682 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb611f-c943-4580-b7ec-8a987962bb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:04.149952 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb611f-c943-4580-b7ec-8a987962bb18 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:04.150728 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bb611f-c943-4580-b7ec-8a987962bb18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:04.152516 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bb611f-c943-4580-b7ec-8a987962bb18 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:04.217189 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 704/1414] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:03 2022] POST /identity/v3/users => generated 287 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:04.218467 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fcce48-30a1-49ec-8719-228e8994052b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:04.248035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fcce48-30a1-49ec-8719-228e8994052b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:04.250128 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb611f-c943-4580-b7ec-8a987962bb18 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:04.252568 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-12bb611f-c943-4580-b7ec-8a987962bb18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:04.273647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9088b959-e392-44fc-b06b-e6e7172980d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:04.274270 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088b959-e392-44fc-b06b-e6e7172980d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:04.276050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088b959-e392-44fc-b06b-e6e7172980d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:04.276050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088b959-e392-44fc-b06b-e6e7172980d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:04.276050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9088b959-e392-44fc-b06b-e6e7172980d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:04.280651 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9088b959-e392-44fc-b06b-e6e7172980d0 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:04.308909 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb611f-c943-4580-b7ec-8a987962bb18 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:04.327490 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003cb6d2-afdb-474f-a375-494cff98a432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:04.327632 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cb6d2-afdb-474f-a375-494cff98a432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:04.327808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cb6d2-afdb-474f-a375-494cff98a432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:04.328083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cb6d2-afdb-474f-a375-494cff98a432 admin admin] PATH_INFO: `/v3/users/cf78f545cf7943a6a24dbfcfd1555443` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:04.329676 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003cb6d2-afdb-474f-a375-494cff98a432 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf78f545cf7943a6a24dbfcfd1555443)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:04.330067 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 711/1415] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:04.330973 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9088b959-e392-44fc-b06b-e6e7172980d0 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:04.337561 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003cb6d2-afdb-474f-a375-494cff98a432 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:04.362334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9088b959-e392-44fc-b06b-e6e7172980d0 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:04.374303 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617cbf4-2562-4d0f-b7de-acecbfbb78a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:04.374937 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617cbf4-2562-4d0f-b7de-acecbfbb78a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:04.375153 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617cbf4-2562-4d0f-b7de-acecbfbb78a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:04.375491 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617cbf4-2562-4d0f-b7de-acecbfbb78a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:04.376112 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617cbf4-2562-4d0f-b7de-acecbfbb78a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:04.377753 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617cbf4-2562-4d0f-b7de-acecbfbb78a7 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:04.438236 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 712/1416] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:04.443911 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617cbf4-2562-4d0f-b7de-acecbfbb78a7 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:04.456906 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617cbf4-2562-4d0f-b7de-acecbfbb78a7 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:04.463817 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-003cb6d2-afdb-474f-a375-494cff98a432 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf78f545cf7943a6a24dbfcfd1555443'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:04.464771 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c037d-0d36-4ab8-8ac9-7229523ade14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:04.465452 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c037d-0d36-4ab8-8ac9-7229523ade14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:04.465728 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c037d-0d36-4ab8-8ac9-7229523ade14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:04.466066 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c037d-0d36-4ab8-8ac9-7229523ade14 admin admin] PATH_INFO: `/v3/projects/c69790f945f0443ab8121ef99b6acef8/users/7bdff535b8924ac8bfa9cb4b2fb1e2f4/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:04.466479 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-003cb6d2-afdb-474f-a375-494cff98a432 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf78f545cf7943a6a24dbfcfd1555443'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:04.494843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-003cb6d2-afdb-474f-a375-494cff98a432 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf78f545cf7943a6a24dbfcfd1555443'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:04.495053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c037d-0d36-4ab8-8ac9-7229523ade14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c69790f945f0443ab8121ef99b6acef8, user_id=7bdff535b8924ac8bfa9cb4b2fb1e2f4, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:04.498782 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c037d-0d36-4ab8-8ac9-7229523ade14 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:04.517672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-003cb6d2-afdb-474f-a375-494cff98a432 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf78f545cf7943a6a24dbfcfd1555443'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:04.527021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-003cb6d2-afdb-474f-a375-494cff98a432 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf78f545cf7943a6a24dbfcfd1555443'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:04.534551 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 705/1417] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:04 2022] DELETE /identity/v3/users/cf78f545cf7943a6a24dbfcfd1555443 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:04.539592 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161a47a-4c20-408f-bdb9-1a49b80d6052 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:04.542757 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 713/1418] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:04 2022] PUT /identity/v3/projects/c69790f945f0443ab8121ef99b6acef8/users/7bdff535b8924ac8bfa9cb4b2fb1e2f4/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:04.543349 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161a47a-4c20-408f-bdb9-1a49b80d6052 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:04.543828 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161a47a-4c20-408f-bdb9-1a49b80d6052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:04.557699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afe5cc-9cf9-45d0-a3a5-4d2bf3344dc5 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:04.568122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afe5cc-9cf9-45d0-a3a5-4d2bf3344dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:04.568401 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afe5cc-9cf9-45d0-a3a5-4d2bf3344dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:04.610720 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-4161a47a-4c20-408f-bdb9-1a49b80d6052 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:04.612160 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-4161a47a-4c20-408f-bdb9-1a49b80d6052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:04.624242 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-88afe5cc-9cf9-45d0-a3a5-4d2bf3344dc5 None None] MFA Rules not processed for user `7bdff535b8924ac8bfa9cb4b2fb1e2f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:04.625688 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-88afe5cc-9cf9-45d0-a3a5-4d2bf3344dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:04.645781 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:04.743149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 714/1419] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:04.755095 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afe5cc-9cf9-45d0-a3a5-4d2bf3344dc5 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:04.756813 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-88afe5cc-9cf9-45d0-a3a5-4d2bf3344dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:04.773872 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 706/1420] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:04.792441 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161a47a-4c20-408f-bdb9-1a49b80d6052 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:04.794037 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-4161a47a-4c20-408f-bdb9-1a49b80d6052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:04.837745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:04.853468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afe5cc-9cf9-45d0-a3a5-4d2bf3344dc5 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:04.857978 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7563503-c8c8-43d8-9fef-fbf1d32cb5ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:04.858724 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7563503-c8c8-43d8-9fef-fbf1d32cb5ac service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:04.859008 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7563503-c8c8-43d8-9fef-fbf1d32cb5ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:04.859458 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7563503-c8c8-43d8-9fef-fbf1d32cb5ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:04.869331 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7563503-c8c8-43d8-9fef-fbf1d32cb5ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:04.871041 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7563503-c8c8-43d8-9fef-fbf1d32cb5ac service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:04.877451 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161a47a-4c20-408f-bdb9-1a49b80d6052 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:04.886356 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 715/1421] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:05:04.888483 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46fc78-79bc-44b0-b8a5-567885c475b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:04.889151 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46fc78-79bc-44b0-b8a5-567885c475b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:04.889430 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46fc78-79bc-44b0-b8a5-567885c475b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:04.889710 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46fc78-79bc-44b0-b8a5-567885c475b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:04.910181 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7563503-c8c8-43d8-9fef-fbf1d32cb5ac service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:04.912443 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b7563503-c8c8-43d8-9fef-fbf1d32cb5ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:04.927970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46fc78-79bc-44b0-b8a5-567885c475b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:04.929874 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46fc78-79bc-44b0-b8a5-567885c475b0 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:04.930567 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7563503-c8c8-43d8-9fef-fbf1d32cb5ac service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:04.936729 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668639a4-bcde-4be6-84cb-634fb5e32ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:04.939473 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668639a4-bcde-4be6-84cb-634fb5e32ea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:04.939473 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668639a4-bcde-4be6-84cb-634fb5e32ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:04.939473 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668639a4-bcde-4be6-84cb-634fb5e32ea2 admin admin] PATH_INFO: `/v3/users/7bdff535b8924ac8bfa9cb4b2fb1e2f4` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:04.940055 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668639a4-bcde-4be6-84cb-634fb5e32ea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bdff535b8924ac8bfa9cb4b2fb1e2f4)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:04.944813 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668639a4-bcde-4be6-84cb-634fb5e32ea2 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:04.950650 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 707/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:05.039469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-668639a4-bcde-4be6-84cb-634fb5e32ea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bdff535b8924ac8bfa9cb4b2fb1e2f4'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:05.040308 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-668639a4-bcde-4be6-84cb-634fb5e32ea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bdff535b8924ac8bfa9cb4b2fb1e2f4'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:05.054381 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-668639a4-bcde-4be6-84cb-634fb5e32ea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bdff535b8924ac8bfa9cb4b2fb1e2f4'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:05.069526 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-668639a4-bcde-4be6-84cb-634fb5e32ea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bdff535b8924ac8bfa9cb4b2fb1e2f4'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:05.070481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-668639a4-bcde-4be6-84cb-634fb5e32ea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bdff535b8924ac8bfa9cb4b2fb1e2f4'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:05.089566 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 716/1423] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:04 2022] DELETE /identity/v3/users/7bdff535b8924ac8bfa9cb4b2fb1e2f4 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:05.094046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da7140-bd88-436b-bfeb-8ed3b6a567a7 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:05.094336 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da7140-bd88-436b-bfeb-8ed3b6a567a7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:05.094619 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da7140-bd88-436b-bfeb-8ed3b6a567a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:05.137733 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-10da7140-bd88-436b-bfeb-8ed3b6a567a7 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:05.138959 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-10da7140-bd88-436b-bfeb-8ed3b6a567a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:05.255708 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 708/1424] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:05.267721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668639a4-bcde-4be6-84cb-634fb5e32ea2 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:05.268145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-668639a4-bcde-4be6-84cb-634fb5e32ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:05.377427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668639a4-bcde-4be6-84cb-634fb5e32ea2 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:05.384270 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58895263-902b-4584-bdc8-3f761f0e98e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:05.385017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58895263-902b-4584-bdc8-3f761f0e98e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:05.385304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58895263-902b-4584-bdc8-3f761f0e98e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:05.385580 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58895263-902b-4584-bdc8-3f761f0e98e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:05.397390 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58895263-902b-4584-bdc8-3f761f0e98e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:05.399648 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58895263-902b-4584-bdc8-3f761f0e98e7 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:05.413928 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 717/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:05.559736 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da7140-bd88-436b-bfeb-8ed3b6a567a7 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:05.563742 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-10da7140-bd88-436b-bfeb-8ed3b6a567a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:05.579454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da7140-bd88-436b-bfeb-8ed3b6a567a7 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:05.591073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e70907-1b90-4bf3-a645-fcb72dcb1dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:05.630659 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e70907-1b90-4bf3-a645-fcb72dcb1dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:05.630997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e70907-1b90-4bf3-a645-fcb72dcb1dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:05.631272 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e70907-1b90-4bf3-a645-fcb72dcb1dd9 admin admin] PATH_INFO: `/v3/projects/09732b03e24c43a4aaaf4e80fa15dc94` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:05.644760 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e70907-1b90-4bf3-a645-fcb72dcb1dd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09732b03e24c43a4aaaf4e80fa15dc94)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:05.652132 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e70907-1b90-4bf3-a645-fcb72dcb1dd9 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:05.837673 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-72e70907-1b90-4bf3-a645-fcb72dcb1dd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09732b03e24c43a4aaaf4e80fa15dc94', 'request_id': 'req-72e70907-1b90-4bf3-a645-fcb72dcb1dd9'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:05.848348 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:05.852378 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-72e70907-1b90-4bf3-a645-fcb72dcb1dd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09732b03e24c43a4aaaf4e80fa15dc94', 'request_id': 'req-72e70907-1b90-4bf3-a645-fcb72dcb1dd9'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:05.866792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-72e70907-1b90-4bf3-a645-fcb72dcb1dd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09732b03e24c43a4aaaf4e80fa15dc94', 'request_id': 'req-72e70907-1b90-4bf3-a645-fcb72dcb1dd9'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:05.899225 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-72e70907-1b90-4bf3-a645-fcb72dcb1dd9 admin admin] The token cache is being invalidate because project 09732b03e24c43a4aaaf4e80fa15dc94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:05.899831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-72e70907-1b90-4bf3-a645-fcb72dcb1dd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:05.900652 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-72e70907-1b90-4bf3-a645-fcb72dcb1dd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:05.905951 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 709/1426] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:05 2022] DELETE /identity/v3/projects/09732b03e24c43a4aaaf4e80fa15dc94 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:05.906941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58895263-902b-4584-bdc8-3f761f0e98e7 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:05.917742 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58895263-902b-4584-bdc8-3f761f0e98e7 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:05.995887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc4dee-f643-4d49-9b97-e108513fe50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:05.996538 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc4dee-f643-4d49-9b97-e108513fe50d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:05.996892 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc4dee-f643-4d49-9b97-e108513fe50d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:05.997115 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc4dee-f643-4d49-9b97-e108513fe50d admin admin] PATH_INFO: `/v3/users/5a99d9ac88f54f989b2ff9d7496cf583` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:05.998265 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc4dee-f643-4d49-9b97-e108513fe50d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a99d9ac88f54f989b2ff9d7496cf583)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:06.000234 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc4dee-f643-4d49-9b97-e108513fe50d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:06.005533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:06.055949 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a3fc4dee-f643-4d49-9b97-e108513fe50d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a99d9ac88f54f989b2ff9d7496cf583'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:06.056667 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a3fc4dee-f643-4d49-9b97-e108513fe50d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a99d9ac88f54f989b2ff9d7496cf583'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:06.065272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a3fc4dee-f643-4d49-9b97-e108513fe50d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a99d9ac88f54f989b2ff9d7496cf583'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:06.078724 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a3fc4dee-f643-4d49-9b97-e108513fe50d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a99d9ac88f54f989b2ff9d7496cf583'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:06.079373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a3fc4dee-f643-4d49-9b97-e108513fe50d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a99d9ac88f54f989b2ff9d7496cf583'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:06.092406 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 718/1427] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:05 2022] DELETE /identity/v3/users/5a99d9ac88f54f989b2ff9d7496cf583 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:06.103443 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e70907-1b90-4bf3-a645-fcb72dcb1dd9 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:06.104922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-72e70907-1b90-4bf3-a645-fcb72dcb1dd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:06.127645 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e70907-1b90-4bf3-a645-fcb72dcb1dd9 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:06.204955 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ac34b-14df-4d75-af30-7f3c119b928f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:06.205609 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ac34b-14df-4d75-af30-7f3c119b928f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:06.205888 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ac34b-14df-4d75-af30-7f3c119b928f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:06.206171 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ac34b-14df-4d75-af30-7f3c119b928f admin admin] PATH_INFO: `/v3/projects/c69790f945f0443ab8121ef99b6acef8` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:06.208087 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ac34b-14df-4d75-af30-7f3c119b928f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c69790f945f0443ab8121ef99b6acef8)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:06.210523 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ac34b-14df-4d75-af30-7f3c119b928f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:06.260678 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6a5ac34b-14df-4d75-af30-7f3c119b928f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c69790f945f0443ab8121ef99b6acef8', 'request_id': 'req-6a5ac34b-14df-4d75-af30-7f3c119b928f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:06.270705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6a5ac34b-14df-4d75-af30-7f3c119b928f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c69790f945f0443ab8121ef99b6acef8', 'request_id': 'req-6a5ac34b-14df-4d75-af30-7f3c119b928f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:06.279857 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6a5ac34b-14df-4d75-af30-7f3c119b928f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c69790f945f0443ab8121ef99b6acef8', 'request_id': 'req-6a5ac34b-14df-4d75-af30-7f3c119b928f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:06.305941 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6a5ac34b-14df-4d75-af30-7f3c119b928f admin admin] The token cache is being invalidate because project c69790f945f0443ab8121ef99b6acef8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:06.306372 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6a5ac34b-14df-4d75-af30-7f3c119b928f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:06.307130 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6a5ac34b-14df-4d75-af30-7f3c119b928f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:06.313095 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 710/1428] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:06 2022] DELETE /identity/v3/projects/c69790f945f0443ab8121ef99b6acef8 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:06.315112 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b607f-6941-4c42-baba-206b605348f6 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:06.315373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b607f-6941-4c42-baba-206b605348f6 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:06.316030 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b607f-6941-4c42-baba-206b605348f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:06.342825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-3d3b607f-6941-4c42-baba-206b605348f6 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:06.344091 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3d3b607f-6941-4c42-baba-206b605348f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:06.408940 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 719/1429] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:06.413832 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ac34b-14df-4d75-af30-7f3c119b928f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:06.415378 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6a5ac34b-14df-4d75-af30-7f3c119b928f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:06.418088 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede97a8b-43b8-423e-8c91-610125c8d42a None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:06.418336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede97a8b-43b8-423e-8c91-610125c8d42a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:06.418621 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede97a8b-43b8-423e-8c91-610125c8d42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:06.431122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ac34b-14df-4d75-af30-7f3c119b928f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:06.435638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a9828-b048-4cb0-9c13-379bf8466a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:06.436273 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a9828-b048-4cb0-9c13-379bf8466a5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:06.436552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a9828-b048-4cb0-9c13-379bf8466a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:06.436857 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a9828-b048-4cb0-9c13-379bf8466a5d admin admin] PATH_INFO: `/v3/projects/43cbf0d29a0243d498a7ed1acac305f3` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:06.437990 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a9828-b048-4cb0-9c13-379bf8466a5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=43cbf0d29a0243d498a7ed1acac305f3)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:06.439897 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a9828-b048-4cb0-9c13-379bf8466a5d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:06.446370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-ede97a8b-43b8-423e-8c91-610125c8d42a None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:06.447639 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ede97a8b-43b8-423e-8c91-610125c8d42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:06.455588 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 720/1430] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:06.460045 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede97a8b-43b8-423e-8c91-610125c8d42a None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:06.469679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede97a8b-43b8-423e-8c91-610125c8d42a None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:06.475208 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b5ba0-f0ed-4814-b82e-b1b7eecbbe80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:06.475811 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b5ba0-f0ed-4814-b82e-b1b7eecbbe80 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:06.476082 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b5ba0-f0ed-4814-b82e-b1b7eecbbe80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:06.476345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b5ba0-f0ed-4814-b82e-b1b7eecbbe80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:06.477018 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b5ba0-f0ed-4814-b82e-b1b7eecbbe80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:05:06.477318 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b5ba0-f0ed-4814-b82e-b1b7eecbbe80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:06.478928 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b5ba0-f0ed-4814-b82e-b1b7eecbbe80 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:06.491558 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4a3a9828-b048-4cb0-9c13-379bf8466a5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43cbf0d29a0243d498a7ed1acac305f3', 'request_id': 'req-4a3a9828-b048-4cb0-9c13-379bf8466a5d'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:06.491944 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 721/1431] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 06:05:06.496212 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b5ba0-f0ed-4814-b82e-b1b7eecbbe80 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:06.496980 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4a3a9828-b048-4cb0-9c13-379bf8466a5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43cbf0d29a0243d498a7ed1acac305f3', 'request_id': 'req-4a3a9828-b048-4cb0-9c13-379bf8466a5d'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:06.503431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4a3a9828-b048-4cb0-9c13-379bf8466a5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43cbf0d29a0243d498a7ed1acac305f3', 'request_id': 'req-4a3a9828-b048-4cb0-9c13-379bf8466a5d'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:06.506172 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b5ba0-f0ed-4814-b82e-b1b7eecbbe80 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:06.513242 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=82,keystone:INSERT=3,keystone:DELETE=15 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:06.518871 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4a3a9828-b048-4cb0-9c13-379bf8466a5d admin admin] The token cache is being invalidate because project 43cbf0d29a0243d498a7ed1acac305f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:06.519429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4a3a9828-b048-4cb0-9c13-379bf8466a5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:06.520557 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4a3a9828-b048-4cb0-9c13-379bf8466a5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:06.522273 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 711/1432] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:06 2022] DELETE /identity/v3/projects/43cbf0d29a0243d498a7ed1acac305f3 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:06.526940 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fab15-e897-48b3-a2b0-a1d8a2cd9a49 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:06.527221 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fab15-e897-48b3-a2b0-a1d8a2cd9a49 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:06.527525 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fab15-e897-48b3-a2b0-a1d8a2cd9a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:06.547580 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-298fab15-e897-48b3-a2b0-a1d8a2cd9a49 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:06.548834 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-298fab15-e897-48b3-a2b0-a1d8a2cd9a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:06.569299 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b92acb-aa42-4145-b57a-fb8a3c3c6e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:06.569909 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b92acb-aa42-4145-b57a-fb8a3c3c6e8c admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:06.570188 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b92acb-aa42-4145-b57a-fb8a3c3c6e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:06.570462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b92acb-aa42-4145-b57a-fb8a3c3c6e8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:06.571084 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b92acb-aa42-4145-b57a-fb8a3c3c6e8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:06.572994 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b92acb-aa42-4145-b57a-fb8a3c3c6e8c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:06.582524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:06.602238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 722/1433] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:06 2022] POST /identity/v3/projects => generated 361 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:06.606483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b92acb-aa42-4145-b57a-fb8a3c3c6e8c admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:06.608176 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-99b92acb-aa42-4145-b57a-fb8a3c3c6e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:06.615912 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 712/1434] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:05:06.624343 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9a9e0-80c2-4bcf-8373-f50b0f06161e None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:06.624626 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9a9e0-80c2-4bcf-8373-f50b0f06161e None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:06.624924 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9a9e0-80c2-4bcf-8373-f50b0f06161e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:06.645248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-a7b9a9e0-80c2-4bcf-8373-f50b0f06161e None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:06.646788 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a7b9a9e0-80c2-4bcf-8373-f50b0f06161e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:06.677356 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b92acb-aa42-4145-b57a-fb8a3c3c6e8c admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:06.685802 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85976502-2ef4-4de0-a1cc-45cab9265481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:06.686593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85976502-2ef4-4de0-a1cc-45cab9265481 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:06.687110 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85976502-2ef4-4de0-a1cc-45cab9265481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:06.687110 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85976502-2ef4-4de0-a1cc-45cab9265481 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:06.694168 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85976502-2ef4-4de0-a1cc-45cab9265481 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:06.694168 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85976502-2ef4-4de0-a1cc-45cab9265481 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:06.724756 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 723/1435] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:06 2022] POST /identity/v3/users => generated 281 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:05:06.728936 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85976502-2ef4-4de0-a1cc-45cab9265481 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:06.730239 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 713/1436] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:05:06.731160 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9a9e0-80c2-4bcf-8373-f50b0f06161e None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:06.740359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85976502-2ef4-4de0-a1cc-45cab9265481 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:06.740556 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9a9e0-80c2-4bcf-8373-f50b0f06161e None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:06.746551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca43688-174e-4256-a388-21cb8c8e7085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:06.747245 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca43688-174e-4256-a388-21cb8c8e7085 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:06.747534 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e48af-afeb-4ba1-b981-781674861b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:06.747668 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca43688-174e-4256-a388-21cb8c8e7085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:06.747925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca43688-174e-4256-a388-21cb8c8e7085 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:06.748096 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e48af-afeb-4ba1-b981-781674861b1f admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:06.748379 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e48af-afeb-4ba1-b981-781674861b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:06.748500 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca43688-174e-4256-a388-21cb8c8e7085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:06.748659 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e48af-afeb-4ba1-b981-781674861b1f admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:06.749325 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e48af-afeb-4ba1-b981-781674861b1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:05:06.749628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e48af-afeb-4ba1-b981-781674861b1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:06.750241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca43688-174e-4256-a388-21cb8c8e7085 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:06.751222 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e48af-afeb-4ba1-b981-781674861b1f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:06.763088 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 724/1437] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:06.767088 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 714/1438] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:05:06.768148 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca43688-174e-4256-a388-21cb8c8e7085 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:06.772301 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e48af-afeb-4ba1-b981-781674861b1f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:06.787301 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca43688-174e-4256-a388-21cb8c8e7085 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:06.792624 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e48af-afeb-4ba1-b981-781674861b1f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:06.793874 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794a6d3-980a-4781-bcfa-9f82cbf8b797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:06.794936 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794a6d3-980a-4781-bcfa-9f82cbf8b797 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:06.795499 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794a6d3-980a-4781-bcfa-9f82cbf8b797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:06.796387 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794a6d3-980a-4781-bcfa-9f82cbf8b797 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:06.797738 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d794a6d3-980a-4781-bcfa-9f82cbf8b797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:06.798792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a9311-3d80-4dd7-9934-e22e923c7e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:06.799501 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a9311-3d80-4dd7-9934-e22e923c7e9f admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:06.800022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a9311-3d80-4dd7-9934-e22e923c7e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:06.800363 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a9311-3d80-4dd7-9934-e22e923c7e9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:06.801397 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d794a6d3-980a-4781-bcfa-9f82cbf8b797 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:06.802085 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a9311-3d80-4dd7-9934-e22e923c7e9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:06.805193 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a9311-3d80-4dd7-9934-e22e923c7e9f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:06.819986 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 725/1439] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:06.820892 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794a6d3-980a-4781-bcfa-9f82cbf8b797 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:06.840462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794a6d3-980a-4781-bcfa-9f82cbf8b797 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:06.847272 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 715/1440] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:06 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:06.849097 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d141809-566a-4028-8901-394813f3a22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:06.849496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d141809-566a-4028-8901-394813f3a22b admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:06.849850 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d141809-566a-4028-8901-394813f3a22b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:06.850195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d141809-566a-4028-8901-394813f3a22b admin admin] PATH_INFO: `/v3/projects/0de28afe5e654d3599266e7fe33ffe9a/users/da84e7f592d14980b2d78392f8143770/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:06.860809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a9311-3d80-4dd7-9934-e22e923c7e9f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:06.869958 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d141809-566a-4028-8901-394813f3a22b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de28afe5e654d3599266e7fe33ffe9a, user_id=da84e7f592d14980b2d78392f8143770, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:06.873132 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d141809-566a-4028-8901-394813f3a22b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:06.910937 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 726/1441] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:06 2022] PUT /identity/v3/projects/0de28afe5e654d3599266e7fe33ffe9a/users/da84e7f592d14980b2d78392f8143770/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:06.926238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7b141-bed2-41a7-b36f-c42a9b159dcf None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:06.927702 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7b141-bed2-41a7-b36f-c42a9b159dcf None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:06.928197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7b141-bed2-41a7-b36f-c42a9b159dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:06.953933 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a9311-3d80-4dd7-9934-e22e923c7e9f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:06.966005 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:06.975638 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-37c7b141-bed2-41a7-b36f-c42a9b159dcf None None] MFA Rules not processed for user `da84e7f592d14980b2d78392f8143770`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:06.977076 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-37c7b141-bed2-41a7-b36f-c42a9b159dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:07.030825 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819de461-30e4-4bb7-bd8e-8a724dbe3566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:07.031903 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819de461-30e4-4bb7-bd8e-8a724dbe3566 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:07.032221 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819de461-30e4-4bb7-bd8e-8a724dbe3566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:07.032655 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819de461-30e4-4bb7-bd8e-8a724dbe3566 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:07.033340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819de461-30e4-4bb7-bd8e-8a724dbe3566 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:07.037618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819de461-30e4-4bb7-bd8e-8a724dbe3566 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:07.078023 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 716/1442] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:06 2022] POST /identity/v3/users => generated 289 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:07.079264 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819de461-30e4-4bb7-bd8e-8a724dbe3566 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:07.102813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819de461-30e4-4bb7-bd8e-8a724dbe3566 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:07.114573 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae748a-b5c2-4389-9435-dce46f1c5604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:07.115394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae748a-b5c2-4389-9435-dce46f1c5604 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:07.115715 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae748a-b5c2-4389-9435-dce46f1c5604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:07.116351 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae748a-b5c2-4389-9435-dce46f1c5604 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:07.117189 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae748a-b5c2-4389-9435-dce46f1c5604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:07.118855 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae748a-b5c2-4389-9435-dce46f1c5604 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:07.135240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 717/1443] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:07.136832 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae748a-b5c2-4389-9435-dce46f1c5604 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:07.146055 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 727/1444] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:07.148579 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae748a-b5c2-4389-9435-dce46f1c5604 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:07.153775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d1fe1-0f6d-4dc0-8f1f-3c2203d17442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:07.154393 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d1fe1-0f6d-4dc0-8f1f-3c2203d17442 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:07.154688 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d1fe1-0f6d-4dc0-8f1f-3c2203d17442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:07.155118 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d1fe1-0f6d-4dc0-8f1f-3c2203d17442 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:07.155779 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d1fe1-0f6d-4dc0-8f1f-3c2203d17442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:07.157427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d1fe1-0f6d-4dc0-8f1f-3c2203d17442 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:07.173100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 718/1445] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:07.178018 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7b141-bed2-41a7-b36f-c42a9b159dcf None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:07.189580 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d1fe1-0f6d-4dc0-8f1f-3c2203d17442 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:07.190807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7b141-bed2-41a7-b36f-c42a9b159dcf None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:07.191476 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fd4d1fe1-0f6d-4dc0-8f1f-3c2203d17442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:07.195890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bda006-8cf8-42be-af27-28b7a274be0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:07.196641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bda006-8cf8-42be-af27-28b7a274be0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:07.197023 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bda006-8cf8-42be-af27-28b7a274be0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:07.197557 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bda006-8cf8-42be-af27-28b7a274be0c admin admin] PATH_INFO: `/v3/projects/e7a51dfdf47046cbbf9434a7b0e542ad/users/fe6e30e58fbf4fdda4d1f65026238466/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:07.214307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bda006-8cf8-42be-af27-28b7a274be0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7a51dfdf47046cbbf9434a7b0e542ad, user_id=fe6e30e58fbf4fdda4d1f65026238466, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:07.217109 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bda006-8cf8-42be-af27-28b7a274be0c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:07.227923 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:07.241809 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 728/1446] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:07 2022] PUT /identity/v3/projects/e7a51dfdf47046cbbf9434a7b0e542ad/users/fe6e30e58fbf4fdda4d1f65026238466/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:05:07.249828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92c565-2c36-49ce-9370-514efd8fef68 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:07.251002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92c565-2c36-49ce-9370-514efd8fef68 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:07.251691 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92c565-2c36-49ce-9370-514efd8fef68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:07.275826 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d1fe1-0f6d-4dc0-8f1f-3c2203d17442 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:07.283930 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-1a92c565-2c36-49ce-9370-514efd8fef68 None None] MFA Rules not processed for user `fe6e30e58fbf4fdda4d1f65026238466`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:07.285618 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1a92c565-2c36-49ce-9370-514efd8fef68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:07.341633 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a38d0a6-b260-43b9-b45e-461ca8df205d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:07.342749 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38d0a6-b260-43b9-b45e-461ca8df205d service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:07.342749 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38d0a6-b260-43b9-b45e-461ca8df205d service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:07.342938 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38d0a6-b260-43b9-b45e-461ca8df205d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:07.378141 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 729/1447] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:05:07.384862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92c565-2c36-49ce-9370-514efd8fef68 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:07.408467 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:07.426462 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a38d0a6-b260-43b9-b45e-461ca8df205d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:07.429529 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a38d0a6-b260-43b9-b45e-461ca8df205d service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:07.447051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 719/1448] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:05:07.463355 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a38d0a6-b260-43b9-b45e-461ca8df205d service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:07.489613 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92c565-2c36-49ce-9370-514efd8fef68 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:07.496016 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960ef19-1773-4e69-8882-3207255b14f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:07.496659 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960ef19-1773-4e69-8882-3207255b14f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:07.496976 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960ef19-1773-4e69-8882-3207255b14f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:07.497253 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960ef19-1773-4e69-8882-3207255b14f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:07.497898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960ef19-1773-4e69-8882-3207255b14f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:07.501835 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960ef19-1773-4e69-8882-3207255b14f3 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:07.553978 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 730/1449] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:07 2022] POST /identity/v3/projects => generated 377 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:07.557537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960ef19-1773-4e69-8882-3207255b14f3 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:07.582888 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a38d0a6-b260-43b9-b45e-461ca8df205d service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:07.642005 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960ef19-1773-4e69-8882-3207255b14f3 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:07.647427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b382242-ad47-44dc-a3c2-6acba7e0eab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:07.647599 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5900c-feb5-4335-bcc4-670140906635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:07.648159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b382242-ad47-44dc-a3c2-6acba7e0eab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:07.648272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5900c-feb5-4335-bcc4-670140906635 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:07.648520 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b382242-ad47-44dc-a3c2-6acba7e0eab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:07.648619 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5900c-feb5-4335-bcc4-670140906635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:07.648851 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5900c-feb5-4335-bcc4-670140906635 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:07.649025 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b382242-ad47-44dc-a3c2-6acba7e0eab1 admin admin] PATH_INFO: `/v3/users/da84e7f592d14980b2d78392f8143770` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:07.649530 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5900c-feb5-4335-bcc4-670140906635 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:07.650132 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b382242-ad47-44dc-a3c2-6acba7e0eab1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da84e7f592d14980b2d78392f8143770)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:07.651411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5900c-feb5-4335-bcc4-670140906635 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:07.652193 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b382242-ad47-44dc-a3c2-6acba7e0eab1 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:07.676991 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 731/1450] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:07 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:07.681010 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5900c-feb5-4335-bcc4-670140906635 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:07.686264 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:07.692313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5900c-feb5-4335-bcc4-670140906635 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:07.700621 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a6fef-7151-4e57-bf24-a1937540871c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:07.701535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a6fef-7151-4e57-bf24-a1937540871c admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:07.701816 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a6fef-7151-4e57-bf24-a1937540871c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:07.702300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a6fef-7151-4e57-bf24-a1937540871c admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:07.702943 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a6fef-7151-4e57-bf24-a1937540871c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:07.704577 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a6fef-7151-4e57-bf24-a1937540871c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:07.720786 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 732/1451] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:05:07.721692 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a6fef-7151-4e57-bf24-a1937540871c admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:07.723498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9b382242-ad47-44dc-a3c2-6acba7e0eab1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da84e7f592d14980b2d78392f8143770'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:07.724220 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9b382242-ad47-44dc-a3c2-6acba7e0eab1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da84e7f592d14980b2d78392f8143770'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:07.733129 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a6fef-7151-4e57-bf24-a1937540871c admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:07.738092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9b382242-ad47-44dc-a3c2-6acba7e0eab1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da84e7f592d14980b2d78392f8143770'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:07.739217 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a892b-d88e-4e2f-847f-4c0695cc693f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:07.739949 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a892b-d88e-4e2f-847f-4c0695cc693f admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:07.740231 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a892b-d88e-4e2f-847f-4c0695cc693f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:07.740505 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a892b-d88e-4e2f-847f-4c0695cc693f admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:07.741295 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a892b-d88e-4e2f-847f-4c0695cc693f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:07.742997 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a892b-d88e-4e2f-847f-4c0695cc693f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:07.755605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 733/1452] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:07.759648 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9b382242-ad47-44dc-a3c2-6acba7e0eab1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da84e7f592d14980b2d78392f8143770'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:07.767002 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9b382242-ad47-44dc-a3c2-6acba7e0eab1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da84e7f592d14980b2d78392f8143770'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:07.768778 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a892b-d88e-4e2f-847f-4c0695cc693f admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:07.771334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-856a892b-d88e-4e2f-847f-4c0695cc693f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:07.780613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 720/1453] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:07 2022] DELETE /identity/v3/users/da84e7f592d14980b2d78392f8143770 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:07.782464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609c2b4-5210-429b-9fcc-b842e518511c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:07.783964 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609c2b4-5210-429b-9fcc-b842e518511c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:07.784421 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609c2b4-5210-429b-9fcc-b842e518511c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:07.785766 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a892b-d88e-4e2f-847f-4c0695cc693f admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:07.811730 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-e609c2b4-5210-429b-9fcc-b842e518511c None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:07.813261 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e609c2b4-5210-429b-9fcc-b842e518511c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:07.825659 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:07.867721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19cfccb-cc5a-4f1c-96e2-fbc19f883a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:07.868593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cfccb-cc5a-4f1c-96e2-fbc19f883a40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:07.868996 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cfccb-cc5a-4f1c-96e2-fbc19f883a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:07.869371 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cfccb-cc5a-4f1c-96e2-fbc19f883a40 admin admin] PATH_INFO: `/v3/projects/1eae26382d364b6f953220fff26dd46a/users/be8f2b4f304b4bbd83e4d518cf67b93b/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:07.909151 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19cfccb-cc5a-4f1c-96e2-fbc19f883a40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eae26382d364b6f953220fff26dd46a, user_id=be8f2b4f304b4bbd83e4d518cf67b93b, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:07.915082 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19cfccb-cc5a-4f1c-96e2-fbc19f883a40 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:07.961627 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 734/1454] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:07 2022] PUT /identity/v3/projects/1eae26382d364b6f953220fff26dd46a/users/be8f2b4f304b4bbd83e4d518cf67b93b/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:07.963654 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d99b2-f1cf-453a-bef9-6cfc2f2c266c None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:07.963654 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d99b2-f1cf-453a-bef9-6cfc2f2c266c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:07.963654 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d99b2-f1cf-453a-bef9-6cfc2f2c266c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:07.968837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 721/1455] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:07.982042 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609c2b4-5210-429b-9fcc-b842e518511c None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:07.983796 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e609c2b4-5210-429b-9fcc-b842e518511c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:08.039408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-137d99b2-f1cf-453a-bef9-6cfc2f2c266c None None] MFA Rules not processed for user `be8f2b4f304b4bbd83e4d518cf67b93b`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:08.041880 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-137d99b2-f1cf-453a-bef9-6cfc2f2c266c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:08.066781 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609c2b4-5210-429b-9fcc-b842e518511c None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:08.191844 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 735/1456] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:08.194230 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d0a1a-3eef-4535-9888-681c8aebc802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:08.194960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d0a1a-3eef-4535-9888-681c8aebc802 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:08.195248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d0a1a-3eef-4535-9888-681c8aebc802 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:08.196331 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d99b2-f1cf-453a-bef9-6cfc2f2c266c None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:08.196672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d0a1a-3eef-4535-9888-681c8aebc802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:08.295590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:08.325867 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d0a1a-3eef-4535-9888-681c8aebc802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:08.328551 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d0a1a-3eef-4535-9888-681c8aebc802 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:08.333230 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d99b2-f1cf-453a-bef9-6cfc2f2c266c None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:08.338867 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05455e55-c349-4172-84ca-4d41bcea3e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:08.339497 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05455e55-c349-4172-84ca-4d41bcea3e54 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:08.339841 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05455e55-c349-4172-84ca-4d41bcea3e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:08.340047 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05455e55-c349-4172-84ca-4d41bcea3e54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:08.340721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05455e55-c349-4172-84ca-4d41bcea3e54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:08.342677 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05455e55-c349-4172-84ca-4d41bcea3e54 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:08.359805 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 722/1457] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:08.390817 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 736/1458] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:08 2022] POST /identity/v3/projects => generated 373 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:08.396735 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d0a1a-3eef-4535-9888-681c8aebc802 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:08.414275 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d0a1a-3eef-4535-9888-681c8aebc802 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:08.442224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:08.509867 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db9e33c-56c5-4970-ada6-85bd946c3731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:08.510758 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9e33c-56c5-4970-ada6-85bd946c3731 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:08.511229 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9e33c-56c5-4970-ada6-85bd946c3731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:08.511970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9e33c-56c5-4970-ada6-85bd946c3731 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:08.512919 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db9e33c-56c5-4970-ada6-85bd946c3731 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:08.515612 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db9e33c-56c5-4970-ada6-85bd946c3731 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:08.544515 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 723/1459] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:08 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:08.550287 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05455e55-c349-4172-84ca-4d41bcea3e54 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:08.565913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05455e55-c349-4172-84ca-4d41bcea3e54 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:08.573072 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da44efd-ab67-44d5-820d-42bc79cc6234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:08.573919 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da44efd-ab67-44d5-820d-42bc79cc6234 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:08.574820 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da44efd-ab67-44d5-820d-42bc79cc6234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:08.575159 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da44efd-ab67-44d5-820d-42bc79cc6234 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:08.575873 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da44efd-ab67-44d5-820d-42bc79cc6234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:08.579151 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da44efd-ab67-44d5-820d-42bc79cc6234 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:08.596761 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 737/1460] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:08.597945 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db9e33c-56c5-4970-ada6-85bd946c3731 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:08.612478 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db9e33c-56c5-4970-ada6-85bd946c3731 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:08.621065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2fe16a-e849-4d91-8440-b9b82f3c2797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:08.621927 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2fe16a-e849-4d91-8440-b9b82f3c2797 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:08.622287 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2fe16a-e849-4d91-8440-b9b82f3c2797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:08.622641 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2fe16a-e849-4d91-8440-b9b82f3c2797 admin admin] PATH_INFO: `/v3/projects/ea23d93663604875a5aac9a12b97a1ec/users/657f9b98e28d409daa88797f047587b7/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:08.651122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2fe16a-e849-4d91-8440-b9b82f3c2797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea23d93663604875a5aac9a12b97a1ec, user_id=657f9b98e28d409daa88797f047587b7, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:08.655358 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2fe16a-e849-4d91-8440-b9b82f3c2797 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:08.678346 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 724/1461] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:08 2022] PUT /identity/v3/projects/ea23d93663604875a5aac9a12b97a1ec/users/657f9b98e28d409daa88797f047587b7/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:05:08.682063 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da44efd-ab67-44d5-820d-42bc79cc6234 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:08.693983 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da44efd-ab67-44d5-820d-42bc79cc6234 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:08.780248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9fb32-8a49-47bc-8aac-174d891378cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:08.782116 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9fb32-8a49-47bc-8aac-174d891378cd admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:08.782808 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9fb32-8a49-47bc-8aac-174d891378cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:08.783353 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9fb32-8a49-47bc-8aac-174d891378cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:08.784306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9fb32-8a49-47bc-8aac-174d891378cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:08.786878 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9fb32-8a49-47bc-8aac-174d891378cd admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:08.799588 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 738/1462] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:08.804426 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2fe16a-e849-4d91-8440-b9b82f3c2797 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:08.819656 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2fe16a-e849-4d91-8440-b9b82f3c2797 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:08.828497 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ed909-4dbe-42c4-8f46-e213b6c06c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:08.829516 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ed909-4dbe-42c4-8f46-e213b6c06c50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:08.829932 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ed909-4dbe-42c4-8f46-e213b6c06c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:08.830311 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ed909-4dbe-42c4-8f46-e213b6c06c50 admin admin] PATH_INFO: `/v3/projects/ea23d93663604875a5aac9a12b97a1ec/users/657f9b98e28d409daa88797f047587b7/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:08.833040 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ed909-4dbe-42c4-8f46-e213b6c06c50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea23d93663604875a5aac9a12b97a1ec, user_id=657f9b98e28d409daa88797f047587b7, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:08.836000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ed909-4dbe-42c4-8f46-e213b6c06c50 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:08.851831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 725/1463] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:08 2022] PUT /identity/v3/projects/ea23d93663604875a5aac9a12b97a1ec/users/657f9b98e28d409daa88797f047587b7/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:08.863820 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b205eef-9dc8-46fa-8108-fa45b92db620 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:08.864346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b205eef-9dc8-46fa-8108-fa45b92db620 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:08.864738 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b205eef-9dc8-46fa-8108-fa45b92db620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:08.902456 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-3b205eef-9dc8-46fa-8108-fa45b92db620 None None] MFA Rules not processed for user `657f9b98e28d409daa88797f047587b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:08.903886 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3b205eef-9dc8-46fa-8108-fa45b92db620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:09.000209 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 739/1464] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:09.003350 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ed909-4dbe-42c4-8f46-e213b6c06c50 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:09.010979 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6a9ed909-4dbe-42c4-8f46-e213b6c06c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:09.102578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ed909-4dbe-42c4-8f46-e213b6c06c50 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:09.107815 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e60d01-6eab-416a-b7e5-247e3c629038 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:09.108584 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e60d01-6eab-416a-b7e5-247e3c629038 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:09.109058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e60d01-6eab-416a-b7e5-247e3c629038 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:09.109376 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e60d01-6eab-416a-b7e5-247e3c629038 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:09.124242 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e60d01-6eab-416a-b7e5-247e3c629038 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:09.126946 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e60d01-6eab-416a-b7e5-247e3c629038 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:09.141081 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 726/1465] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:09.214530 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b205eef-9dc8-46fa-8108-fa45b92db620 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:09.216338 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3b205eef-9dc8-46fa-8108-fa45b92db620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:09.230859 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b205eef-9dc8-46fa-8108-fa45b92db620 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:09.235608 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42951ef1-0a07-4db1-8f97-e11eb6c05fed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:09.236488 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42951ef1-0a07-4db1-8f97-e11eb6c05fed service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:09.236984 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42951ef1-0a07-4db1-8f97-e11eb6c05fed service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:09.237451 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42951ef1-0a07-4db1-8f97-e11eb6c05fed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:09.240390 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-42951ef1-0a07-4db1-8f97-e11eb6c05fed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:09.308197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:09.347952 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42951ef1-0a07-4db1-8f97-e11eb6c05fed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:09.357250 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42951ef1-0a07-4db1-8f97-e11eb6c05fed service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:09.385545 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 740/1466] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:10.473117 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e60d01-6eab-416a-b7e5-247e3c629038 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:10.474747 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-26e60d01-6eab-416a-b7e5-247e3c629038 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:10.684406 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e60d01-6eab-416a-b7e5-247e3c629038 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:10.711231 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee2a5a-fe33-4b92-ac15-b58be901c4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:10.712292 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee2a5a-fe33-4b92-ac15-b58be901c4d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:10.714542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee2a5a-fe33-4b92-ac15-b58be901c4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:10.715051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee2a5a-fe33-4b92-ac15-b58be901c4d8 admin admin] PATH_INFO: `/v3/projects/0de28afe5e654d3599266e7fe33ffe9a` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:10.717827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee2a5a-fe33-4b92-ac15-b58be901c4d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0de28afe5e654d3599266e7fe33ffe9a)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:10.718730 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee2a5a-fe33-4b92-ac15-b58be901c4d8 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:10.752827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:10.784645 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-23ee2a5a-fe33-4b92-ac15-b58be901c4d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0de28afe5e654d3599266e7fe33ffe9a', 'request_id': 'req-23ee2a5a-fe33-4b92-ac15-b58be901c4d8'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:10.793941 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-23ee2a5a-fe33-4b92-ac15-b58be901c4d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0de28afe5e654d3599266e7fe33ffe9a', 'request_id': 'req-23ee2a5a-fe33-4b92-ac15-b58be901c4d8'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:10.802385 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-23ee2a5a-fe33-4b92-ac15-b58be901c4d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0de28afe5e654d3599266e7fe33ffe9a', 'request_id': 'req-23ee2a5a-fe33-4b92-ac15-b58be901c4d8'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:10.818462 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-23ee2a5a-fe33-4b92-ac15-b58be901c4d8 admin admin] The token cache is being invalidate because project 0de28afe5e654d3599266e7fe33ffe9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:10.818783 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-23ee2a5a-fe33-4b92-ac15-b58be901c4d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:10.821153 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-23ee2a5a-fe33-4b92-ac15-b58be901c4d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:10.827324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 727/1467] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:10 2022] DELETE /identity/v3/projects/0de28afe5e654d3599266e7fe33ffe9a => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:10.828303 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14fbd6-c8f4-4488-b1ac-aa5165a402b5 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:10.828926 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14fbd6-c8f4-4488-b1ac-aa5165a402b5 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:10.828926 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14fbd6-c8f4-4488-b1ac-aa5165a402b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:10.853697 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-bb14fbd6-c8f4-4488-b1ac-aa5165a402b5 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:10.854928 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-bb14fbd6-c8f4-4488-b1ac-aa5165a402b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:10.940295 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 741/1468] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:10.948990 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d5655-069f-4e3c-a00a-11855e8d7a58 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:10.949279 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d5655-069f-4e3c-a00a-11855e8d7a58 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:10.949567 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d5655-069f-4e3c-a00a-11855e8d7a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:10.971808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f41d5655-069f-4e3c-a00a-11855e8d7a58 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:10.973360 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f41d5655-069f-4e3c-a00a-11855e8d7a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:10.981454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 728/1469] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:10.985785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14fbd6-c8f4-4488-b1ac-aa5165a402b5 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:10.995767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14fbd6-c8f4-4488-b1ac-aa5165a402b5 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:11.000596 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131c778-1d31-4271-89a4-8063a3daf72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:11.001267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131c778-1d31-4271-89a4-8063a3daf72d admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:11.001546 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131c778-1d31-4271-89a4-8063a3daf72d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:11.001817 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131c778-1d31-4271-89a4-8063a3daf72d admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:11.002452 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5131c778-1d31-4271-89a4-8063a3daf72d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:05:11.002779 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5131c778-1d31-4271-89a4-8063a3daf72d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:11.004936 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5131c778-1d31-4271-89a4-8063a3daf72d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:11.020948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 742/1470] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:05:11.022212 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d5655-069f-4e3c-a00a-11855e8d7a58 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:11.031728 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d5655-069f-4e3c-a00a-11855e8d7a58 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:11.037209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6fe3d4-c546-4364-aeb2-dddaba409620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:11.037821 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6fe3d4-c546-4364-aeb2-dddaba409620 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:11.038101 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6fe3d4-c546-4364-aeb2-dddaba409620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:11.038403 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6fe3d4-c546-4364-aeb2-dddaba409620 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:11.039054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6fe3d4-c546-4364-aeb2-dddaba409620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:11.041021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6fe3d4-c546-4364-aeb2-dddaba409620 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:11.069595 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 729/1471] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:11 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:05:11.073726 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131c778-1d31-4271-89a4-8063a3daf72d admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:11.083954 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131c778-1d31-4271-89a4-8063a3daf72d admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:11.143492 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a18fb-4a3d-4215-904c-f010e45d24eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:11.144235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a18fb-4a3d-4215-904c-f010e45d24eb admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:11.144570 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a18fb-4a3d-4215-904c-f010e45d24eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:11.144878 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a18fb-4a3d-4215-904c-f010e45d24eb admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:11.145499 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a18fb-4a3d-4215-904c-f010e45d24eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:11.147396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a18fb-4a3d-4215-904c-f010e45d24eb admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:11.171284 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 743/1472] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:11 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:11.180873 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6fe3d4-c546-4364-aeb2-dddaba409620 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:11.193254 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6fe3d4-c546-4364-aeb2-dddaba409620 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:11.199892 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7d709-1010-4dba-841b-1ef66f8c0fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:11.200216 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7d709-1010-4dba-841b-1ef66f8c0fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:11.200568 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7d709-1010-4dba-841b-1ef66f8c0fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:11.200952 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7d709-1010-4dba-841b-1ef66f8c0fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:11.201612 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7d709-1010-4dba-841b-1ef66f8c0fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:11.203605 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7d709-1010-4dba-841b-1ef66f8c0fa2 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:11.219144 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 730/1473] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:05:11.222820 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a18fb-4a3d-4215-904c-f010e45d24eb admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:11.233919 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a18fb-4a3d-4215-904c-f010e45d24eb admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:11.238943 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5981b6c-5b4d-475e-acc4-4a2d51d2b24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:11.239554 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5981b6c-5b4d-475e-acc4-4a2d51d2b24e admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:11.239981 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5981b6c-5b4d-475e-acc4-4a2d51d2b24e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:11.240260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5981b6c-5b4d-475e-acc4-4a2d51d2b24e admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:11.240933 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5981b6c-5b4d-475e-acc4-4a2d51d2b24e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:11.244065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5981b6c-5b4d-475e-acc4-4a2d51d2b24e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:11.302726 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 744/1474] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:05:11.310488 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7d709-1010-4dba-841b-1ef66f8c0fa2 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:11.346513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7d709-1010-4dba-841b-1ef66f8c0fa2 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:11.354264 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9201b9c-2e05-4e4b-9c96-8608322a6db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:11.355008 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9201b9c-2e05-4e4b-9c96-8608322a6db2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:11.355295 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9201b9c-2e05-4e4b-9c96-8608322a6db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:11.355566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9201b9c-2e05-4e4b-9c96-8608322a6db2 admin admin] PATH_INFO: `/v3/projects/a0c15dd0961a47e6a5862032515ef1f8/users/5f5c5bb8691648949ebe9ad8445a631c/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:11.378166 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9201b9c-2e05-4e4b-9c96-8608322a6db2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0c15dd0961a47e6a5862032515ef1f8, user_id=5f5c5bb8691648949ebe9ad8445a631c, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:11.382123 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9201b9c-2e05-4e4b-9c96-8608322a6db2 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:11.412959 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 731/1475] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:11 2022] PUT /identity/v3/projects/a0c15dd0961a47e6a5862032515ef1f8/users/5f5c5bb8691648949ebe9ad8445a631c/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:11.424758 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94dc9a9-1dff-48cd-9198-f86b9c2d0ecf None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:11.424758 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94dc9a9-1dff-48cd-9198-f86b9c2d0ecf None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:11.425010 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94dc9a9-1dff-48cd-9198-f86b9c2d0ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:11.466365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d94dc9a9-1dff-48cd-9198-f86b9c2d0ecf None None] MFA Rules not processed for user `5f5c5bb8691648949ebe9ad8445a631c`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:11.469412 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d94dc9a9-1dff-48cd-9198-f86b9c2d0ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:11.542735 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:11.582369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 745/1476] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:11.594498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9201b9c-2e05-4e4b-9c96-8608322a6db2 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:11.596905 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c9201b9c-2e05-4e4b-9c96-8608322a6db2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:11.710100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9201b9c-2e05-4e4b-9c96-8608322a6db2 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:11.716215 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880042fc-7551-4124-9edf-8a4849594824 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:11.717295 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880042fc-7551-4124-9edf-8a4849594824 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:11.717753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880042fc-7551-4124-9edf-8a4849594824 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:11.718179 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880042fc-7551-4124-9edf-8a4849594824 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:11.731697 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880042fc-7551-4124-9edf-8a4849594824 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:11.733952 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880042fc-7551-4124-9edf-8a4849594824 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:11.758270 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 732/1477] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:12.000300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0491cd-bd06-4969-8c3e-234e0c6ebb80 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:12.000826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0491cd-bd06-4969-8c3e-234e0c6ebb80 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:12.001147 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0491cd-bd06-4969-8c3e-234e0c6ebb80 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:12.003169 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 746/1478] 10.222.0.20 () {54 vars in 921 bytes} [Thu Oct 20 06:05:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 06:05:12.025523 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bddc12-2604-4895-a9a8-cd09333f53a0 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:12.026194 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bddc12-2604-4895-a9a8-cd09333f53a0 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:12.026667 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bddc12-2604-4895-a9a8-cd09333f53a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:12.087294 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-b5bddc12-2604-4895-a9a8-cd09333f53a0 None None] MFA Rules not processed for user `ae2c1c886cd94d9f90082f66e7e8c9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:12.091250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b5bddc12-2604-4895-a9a8-cd09333f53a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:12.199928 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:12.265071 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 733/1479] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Oct 20 06:05:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:05:12.314798 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0491cd-bd06-4969-8c3e-234e0c6ebb80 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:12.317880 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-dd0491cd-bd06-4969-8c3e-234e0c6ebb80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:12.342268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0491cd-bd06-4969-8c3e-234e0c6ebb80 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:12.351343 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2eb107-1d52-471a-9dce-93ec25382843 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:12.352343 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2eb107-1d52-471a-9dce-93ec25382843 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:12.352632 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2eb107-1d52-471a-9dce-93ec25382843 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:12.353231 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2eb107-1d52-471a-9dce-93ec25382843 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:12.366031 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2eb107-1d52-471a-9dce-93ec25382843 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:12.367980 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2eb107-1d52-471a-9dce-93ec25382843 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:12.380270 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 747/1480] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:13.241918 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bddc12-2604-4895-a9a8-cd09333f53a0 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:13.243662 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b5bddc12-2604-4895-a9a8-cd09333f53a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:13.375787 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bddc12-2604-4895-a9a8-cd09333f53a0 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:13.386035 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3486a1e-3e2d-480c-8528-810374cb9c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:13.386924 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3486a1e-3e2d-480c-8528-810374cb9c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:13.387361 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3486a1e-3e2d-480c-8528-810374cb9c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:13.387904 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3486a1e-3e2d-480c-8528-810374cb9c55 admin admin] PATH_INFO: `/v3/users/fe6e30e58fbf4fdda4d1f65026238466` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:13.389569 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3486a1e-3e2d-480c-8528-810374cb9c55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe6e30e58fbf4fdda4d1f65026238466)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:13.403282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3486a1e-3e2d-480c-8528-810374cb9c55 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:13.469401 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a3486a1e-3e2d-480c-8528-810374cb9c55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe6e30e58fbf4fdda4d1f65026238466'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:13.470348 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a3486a1e-3e2d-480c-8528-810374cb9c55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe6e30e58fbf4fdda4d1f65026238466'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:13.480550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a3486a1e-3e2d-480c-8528-810374cb9c55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe6e30e58fbf4fdda4d1f65026238466'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:13.512743 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a3486a1e-3e2d-480c-8528-810374cb9c55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe6e30e58fbf4fdda4d1f65026238466'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:13.521958 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a3486a1e-3e2d-480c-8528-810374cb9c55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe6e30e58fbf4fdda4d1f65026238466'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:13.532636 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 734/1481] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:13 2022] DELETE /identity/v3/users/fe6e30e58fbf4fdda4d1f65026238466 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:13.537496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600346a4-aeb8-4f1b-b337-c1d66718b4a5 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:13.538661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600346a4-aeb8-4f1b-b337-c1d66718b4a5 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:13.539244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600346a4-aeb8-4f1b-b337-c1d66718b4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:13.573775 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-600346a4-aeb8-4f1b-b337-c1d66718b4a5 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:13.576745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-600346a4-aeb8-4f1b-b337-c1d66718b4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:13.760988 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 748/1482] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:13.823998 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3486a1e-3e2d-480c-8528-810374cb9c55 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:13.825932 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a3486a1e-3e2d-480c-8528-810374cb9c55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:13.930563 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3486a1e-3e2d-480c-8528-810374cb9c55 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:13.936964 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4ff9b-bf45-4c0d-b915-7ca28e254f73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:13.938155 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4ff9b-bf45-4c0d-b915-7ca28e254f73 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:13.938633 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4ff9b-bf45-4c0d-b915-7ca28e254f73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:13.939098 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4ff9b-bf45-4c0d-b915-7ca28e254f73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:13.971535 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4ff9b-bf45-4c0d-b915-7ca28e254f73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:13.979413 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4ff9b-bf45-4c0d-b915-7ca28e254f73 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:13.994296 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:14.010995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 735/1483] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:15.124579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600346a4-aeb8-4f1b-b337-c1d66718b4a5 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:15.126090 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-600346a4-aeb8-4f1b-b337-c1d66718b4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:15.137944 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600346a4-aeb8-4f1b-b337-c1d66718b4a5 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:15.142426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a73ef-85a7-4e0f-bac1-62d173a04f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:15.143051 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a73ef-85a7-4e0f-bac1-62d173a04f3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:15.143321 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a73ef-85a7-4e0f-bac1-62d173a04f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:15.143585 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a73ef-85a7-4e0f-bac1-62d173a04f3d admin admin] PATH_INFO: `/v3/projects/e7a51dfdf47046cbbf9434a7b0e542ad` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:15.146009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a73ef-85a7-4e0f-bac1-62d173a04f3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7a51dfdf47046cbbf9434a7b0e542ad)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:15.147299 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a73ef-85a7-4e0f-bac1-62d173a04f3d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:15.208404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-323a73ef-85a7-4e0f-bac1-62d173a04f3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7a51dfdf47046cbbf9434a7b0e542ad', 'request_id': 'req-323a73ef-85a7-4e0f-bac1-62d173a04f3d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:15.217066 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-323a73ef-85a7-4e0f-bac1-62d173a04f3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7a51dfdf47046cbbf9434a7b0e542ad', 'request_id': 'req-323a73ef-85a7-4e0f-bac1-62d173a04f3d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:15.225150 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-323a73ef-85a7-4e0f-bac1-62d173a04f3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7a51dfdf47046cbbf9434a7b0e542ad', 'request_id': 'req-323a73ef-85a7-4e0f-bac1-62d173a04f3d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:15.244713 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-323a73ef-85a7-4e0f-bac1-62d173a04f3d admin admin] The token cache is being invalidate because project e7a51dfdf47046cbbf9434a7b0e542ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:15.245113 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-323a73ef-85a7-4e0f-bac1-62d173a04f3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:15.245833 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-323a73ef-85a7-4e0f-bac1-62d173a04f3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:15.247593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 749/1484] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:15 2022] DELETE /identity/v3/projects/e7a51dfdf47046cbbf9434a7b0e542ad => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:15.251901 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4ff9b-bf45-4c0d-b915-7ca28e254f73 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:15.266511 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4ff9b-bf45-4c0d-b915-7ca28e254f73 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:15.329599 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa9e11a-5f51-4ff4-a125-148c62485814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:15.330246 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9e11a-5f51-4ff4-a125-148c62485814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:15.330537 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9e11a-5f51-4ff4-a125-148c62485814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:15.330813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9e11a-5f51-4ff4-a125-148c62485814 admin admin] PATH_INFO: `/v3/users/be8f2b4f304b4bbd83e4d518cf67b93b` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:15.332467 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa9e11a-5f51-4ff4-a125-148c62485814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be8f2b4f304b4bbd83e4d518cf67b93b)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:15.334560 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa9e11a-5f51-4ff4-a125-148c62485814 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:15.382973 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-efa9e11a-5f51-4ff4-a125-148c62485814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be8f2b4f304b4bbd83e4d518cf67b93b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:15.383622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-efa9e11a-5f51-4ff4-a125-148c62485814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be8f2b4f304b4bbd83e4d518cf67b93b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:15.391048 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-efa9e11a-5f51-4ff4-a125-148c62485814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be8f2b4f304b4bbd83e4d518cf67b93b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:15.404497 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-efa9e11a-5f51-4ff4-a125-148c62485814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be8f2b4f304b4bbd83e4d518cf67b93b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:15.411485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-efa9e11a-5f51-4ff4-a125-148c62485814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be8f2b4f304b4bbd83e4d518cf67b93b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:15.417732 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 736/1485] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:15 2022] DELETE /identity/v3/users/be8f2b4f304b4bbd83e4d518cf67b93b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:15.567208 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a73ef-85a7-4e0f-bac1-62d173a04f3d admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:15.568665 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-323a73ef-85a7-4e0f-bac1-62d173a04f3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:15.585144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a73ef-85a7-4e0f-bac1-62d173a04f3d admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:15.644677 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:15.667776 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a634030-a57d-4bcd-a9ae-d79314f66e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:15.668498 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a634030-a57d-4bcd-a9ae-d79314f66e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:15.668877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a634030-a57d-4bcd-a9ae-d79314f66e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:15.669149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a634030-a57d-4bcd-a9ae-d79314f66e7c admin admin] PATH_INFO: `/v3/users/5f5c5bb8691648949ebe9ad8445a631c` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:15.670817 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a634030-a57d-4bcd-a9ae-d79314f66e7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f5c5bb8691648949ebe9ad8445a631c)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:15.672996 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a634030-a57d-4bcd-a9ae-d79314f66e7c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:15.741322 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-7a634030-a57d-4bcd-a9ae-d79314f66e7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f5c5bb8691648949ebe9ad8445a631c'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:15.742013 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-7a634030-a57d-4bcd-a9ae-d79314f66e7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f5c5bb8691648949ebe9ad8445a631c'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:15.749762 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-7a634030-a57d-4bcd-a9ae-d79314f66e7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f5c5bb8691648949ebe9ad8445a631c'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:15.764169 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-7a634030-a57d-4bcd-a9ae-d79314f66e7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f5c5bb8691648949ebe9ad8445a631c'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:15.764900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-7a634030-a57d-4bcd-a9ae-d79314f66e7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f5c5bb8691648949ebe9ad8445a631c'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:15.777645 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 750/1486] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:15 2022] DELETE /identity/v3/users/5f5c5bb8691648949ebe9ad8445a631c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:15.782009 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1536d-f78f-496f-a016-5728fd6e3c78 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:15.782294 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1536d-f78f-496f-a016-5728fd6e3c78 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:15.782563 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1536d-f78f-496f-a016-5728fd6e3c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:15.803285 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f7c1536d-f78f-496f-a016-5728fd6e3c78 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:15.804550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f7c1536d-f78f-496f-a016-5728fd6e3c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:15.808158 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a634030-a57d-4bcd-a9ae-d79314f66e7c admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:15.809687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7a634030-a57d-4bcd-a9ae-d79314f66e7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:15.879313 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 737/1487] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:15.888062 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a634030-a57d-4bcd-a9ae-d79314f66e7c admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:15.889257 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1536d-f78f-496f-a016-5728fd6e3c78 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:15.890927 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f7c1536d-f78f-496f-a016-5728fd6e3c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:15.893179 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5e291-fa23-4540-a90b-1651fcf3c43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:15.894437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5e291-fa23-4540-a90b-1651fcf3c43a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:15.894733 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5e291-fa23-4540-a90b-1651fcf3c43a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:15.895074 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5e291-fa23-4540-a90b-1651fcf3c43a admin admin] PATH_INFO: `/v3/projects/1eae26382d364b6f953220fff26dd46a` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:15.896446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5e291-fa23-4540-a90b-1651fcf3c43a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eae26382d364b6f953220fff26dd46a)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:15.898695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5e291-fa23-4540-a90b-1651fcf3c43a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:15.917768 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:15.953212 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-19d5e291-fa23-4540-a90b-1651fcf3c43a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eae26382d364b6f953220fff26dd46a', 'request_id': 'req-19d5e291-fa23-4540-a90b-1651fcf3c43a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:15.957575 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1536d-f78f-496f-a016-5728fd6e3c78 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:15.960982 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-19d5e291-fa23-4540-a90b-1651fcf3c43a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eae26382d364b6f953220fff26dd46a', 'request_id': 'req-19d5e291-fa23-4540-a90b-1651fcf3c43a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:15.968133 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-19d5e291-fa23-4540-a90b-1651fcf3c43a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eae26382d364b6f953220fff26dd46a', 'request_id': 'req-19d5e291-fa23-4540-a90b-1651fcf3c43a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:15.983037 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-19d5e291-fa23-4540-a90b-1651fcf3c43a admin admin] The token cache is being invalidate because project 1eae26382d364b6f953220fff26dd46a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:15.983313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-19d5e291-fa23-4540-a90b-1651fcf3c43a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:15.983977 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-19d5e291-fa23-4540-a90b-1651fcf3c43a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:15.985806 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 751/1488] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:15 2022] DELETE /identity/v3/projects/1eae26382d364b6f953220fff26dd46a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:15.989996 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5e291-fa23-4540-a90b-1651fcf3c43a admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:15.991401 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-19d5e291-fa23-4540-a90b-1651fcf3c43a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:15.997437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:16.027162 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f6585-8f58-4000-8fa4-d3db329cbff1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:16.027912 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f6585-8f58-4000-8fa4-d3db329cbff1 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:16.028320 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f6585-8f58-4000-8fa4-d3db329cbff1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:16.028840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f6585-8f58-4000-8fa4-d3db329cbff1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:16.031482 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e27f6585-8f58-4000-8fa4-d3db329cbff1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:16.067552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5e291-fa23-4540-a90b-1651fcf3c43a admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:16.072276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7118bc3-7823-4303-af04-b0b12d9f568f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:16.072935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7118bc3-7823-4303-af04-b0b12d9f568f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:16.073221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7118bc3-7823-4303-af04-b0b12d9f568f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:16.073493 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7118bc3-7823-4303-af04-b0b12d9f568f admin admin] PATH_INFO: `/v3/users/657f9b98e28d409daa88797f047587b7` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:16.074727 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7118bc3-7823-4303-af04-b0b12d9f568f admin admin] RBAC: Authorizing `identity:delete_user(user_id=657f9b98e28d409daa88797f047587b7)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:16.076656 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7118bc3-7823-4303-af04-b0b12d9f568f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:16.097369 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f6585-8f58-4000-8fa4-d3db329cbff1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:16.098927 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f6585-8f58-4000-8fa4-d3db329cbff1 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:16.110326 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 738/1489] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:16.126494 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d7118bc3-7823-4303-af04-b0b12d9f568f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '657f9b98e28d409daa88797f047587b7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:16.127163 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d7118bc3-7823-4303-af04-b0b12d9f568f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '657f9b98e28d409daa88797f047587b7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:16.135914 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d7118bc3-7823-4303-af04-b0b12d9f568f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '657f9b98e28d409daa88797f047587b7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:16.149206 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d7118bc3-7823-4303-af04-b0b12d9f568f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '657f9b98e28d409daa88797f047587b7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:16.149848 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d7118bc3-7823-4303-af04-b0b12d9f568f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '657f9b98e28d409daa88797f047587b7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:16.162699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 752/1490] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:15 2022] DELETE /identity/v3/users/657f9b98e28d409daa88797f047587b7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:16.489879 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f6585-8f58-4000-8fa4-d3db329cbff1 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:16.492163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e27f6585-8f58-4000-8fa4-d3db329cbff1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:16.523495 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f6585-8f58-4000-8fa4-d3db329cbff1 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:16.531693 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7118bc3-7823-4303-af04-b0b12d9f568f admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:16.533656 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d7118bc3-7823-4303-af04-b0b12d9f568f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:16.599319 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8871c-9e7d-45b4-82b9-d93383debe39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:16.599961 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8871c-9e7d-45b4-82b9-d93383debe39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:16.600177 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8871c-9e7d-45b4-82b9-d93383debe39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:16.600454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8871c-9e7d-45b4-82b9-d93383debe39 admin admin] PATH_INFO: `/v3/projects/a0c15dd0961a47e6a5862032515ef1f8` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:16.601867 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b8871c-9e7d-45b4-82b9-d93383debe39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0c15dd0961a47e6a5862032515ef1f8)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:16.603792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b8871c-9e7d-45b4-82b9-d93383debe39 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:16.628105 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7118bc3-7823-4303-af04-b0b12d9f568f admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:16.634098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e62c66-c070-40f2-8fb1-a6517e2c305f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:16.634756 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e62c66-c070-40f2-8fb1-a6517e2c305f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:16.635042 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e62c66-c070-40f2-8fb1-a6517e2c305f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:16.635297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e62c66-c070-40f2-8fb1-a6517e2c305f admin admin] PATH_INFO: `/v3/projects/ea23d93663604875a5aac9a12b97a1ec` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:16.636526 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e62c66-c070-40f2-8fb1-a6517e2c305f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea23d93663604875a5aac9a12b97a1ec)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:16.638398 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:16.638550 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e62c66-c070-40f2-8fb1-a6517e2c305f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:16.655502 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-64b8871c-9e7d-45b4-82b9-d93383debe39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0c15dd0961a47e6a5862032515ef1f8', 'request_id': 'req-64b8871c-9e7d-45b4-82b9-d93383debe39'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:16.661723 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-64b8871c-9e7d-45b4-82b9-d93383debe39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0c15dd0961a47e6a5862032515ef1f8', 'request_id': 'req-64b8871c-9e7d-45b4-82b9-d93383debe39'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:16.666007 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:16.668581 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-64b8871c-9e7d-45b4-82b9-d93383debe39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0c15dd0961a47e6a5862032515ef1f8', 'request_id': 'req-64b8871c-9e7d-45b4-82b9-d93383debe39'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:16.681014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-64b8871c-9e7d-45b4-82b9-d93383debe39 admin admin] The token cache is being invalidate because project a0c15dd0961a47e6a5862032515ef1f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:16.681301 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-64b8871c-9e7d-45b4-82b9-d93383debe39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:16.682130 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-64b8871c-9e7d-45b4-82b9-d93383debe39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:16.684031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 739/1491] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:16 2022] DELETE /identity/v3/projects/a0c15dd0961a47e6a5862032515ef1f8 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:16.688368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bf293-77ed-4504-90d3-5c0dd87aef44 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:16.688654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bf293-77ed-4504-90d3-5c0dd87aef44 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:16.688960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bf293-77ed-4504-90d3-5c0dd87aef44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:16.694250 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-90e62c66-c070-40f2-8fb1-a6517e2c305f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea23d93663604875a5aac9a12b97a1ec', 'request_id': 'req-90e62c66-c070-40f2-8fb1-a6517e2c305f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:16.702729 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-90e62c66-c070-40f2-8fb1-a6517e2c305f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea23d93663604875a5aac9a12b97a1ec', 'request_id': 'req-90e62c66-c070-40f2-8fb1-a6517e2c305f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:16.710784 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-765bf293-77ed-4504-90d3-5c0dd87aef44 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:16.711573 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-90e62c66-c070-40f2-8fb1-a6517e2c305f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea23d93663604875a5aac9a12b97a1ec', 'request_id': 'req-90e62c66-c070-40f2-8fb1-a6517e2c305f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:16.712036 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-765bf293-77ed-4504-90d3-5c0dd87aef44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:16.729959 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-90e62c66-c070-40f2-8fb1-a6517e2c305f admin admin] The token cache is being invalidate because project ea23d93663604875a5aac9a12b97a1ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:16.730525 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-90e62c66-c070-40f2-8fb1-a6517e2c305f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:16.731249 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-90e62c66-c070-40f2-8fb1-a6517e2c305f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:16.737307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 753/1492] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:16 2022] DELETE /identity/v3/projects/ea23d93663604875a5aac9a12b97a1ec => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:05:16.739963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3e918-fb6f-4a75-9899-9529977c1882 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:16.741145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3e918-fb6f-4a75-9899-9529977c1882 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:16.741575 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3e918-fb6f-4a75-9899-9529977c1882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:16.773012 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-41a3e918-fb6f-4a75-9899-9529977c1882 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:16.774804 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-41a3e918-fb6f-4a75-9899-9529977c1882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:16.783490 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 740/1493] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:05:16.787426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 754/1494] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:16.796009 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6aa261-e426-493f-96f0-473c06af8d67 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:16.796299 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6aa261-e426-493f-96f0-473c06af8d67 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:16.796577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6aa261-e426-493f-96f0-473c06af8d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:16.796726 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe0663-dcdd-46ea-b9ba-ace4c844a03f None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:16.797018 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe0663-dcdd-46ea-b9ba-ace4c844a03f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:16.797290 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe0663-dcdd-46ea-b9ba-ace4c844a03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:16.820770 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-4c6aa261-e426-493f-96f0-473c06af8d67 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:16.821968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-4c6aa261-e426-493f-96f0-473c06af8d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:16.823366 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-79fe0663-dcdd-46ea-b9ba-ace4c844a03f None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:16.824679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-79fe0663-dcdd-46ea-b9ba-ace4c844a03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:16.830358 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 741/1495] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:16.835493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6aa261-e426-493f-96f0-473c06af8d67 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:16.839474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 755/1496] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:16.844033 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe0663-dcdd-46ea-b9ba-ace4c844a03f None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:16.848639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6aa261-e426-493f-96f0-473c06af8d67 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:16.854088 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9726f30-6266-4188-9394-93bbd0951363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:16.854810 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe0663-dcdd-46ea-b9ba-ace4c844a03f None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:16.855186 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9726f30-6266-4188-9394-93bbd0951363 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:16.855728 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9726f30-6266-4188-9394-93bbd0951363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:16.856209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9726f30-6266-4188-9394-93bbd0951363 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:16.857106 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9726f30-6266-4188-9394-93bbd0951363 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:05:16.857543 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9726f30-6266-4188-9394-93bbd0951363 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:16.859504 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9726f30-6266-4188-9394-93bbd0951363 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:16.860080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4820abc-6bb3-42bb-b230-bb2b5b66acb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:16.860825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4820abc-6bb3-42bb-b230-bb2b5b66acb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:16.861172 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4820abc-6bb3-42bb-b230-bb2b5b66acb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:16.861759 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4820abc-6bb3-42bb-b230-bb2b5b66acb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:16.862478 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4820abc-6bb3-42bb-b230-bb2b5b66acb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:05:16.864686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4820abc-6bb3-42bb-b230-bb2b5b66acb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:16.866402 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4820abc-6bb3-42bb-b230-bb2b5b66acb9 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:16.876139 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 742/1497] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 06:05:16.880800 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9726f30-6266-4188-9394-93bbd0951363 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:16.890439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 756/1498] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:05:16.894839 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9726f30-6266-4188-9394-93bbd0951363 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:16.899973 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532bb8a-82ab-4f49-8295-b48a5c635e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:16.900650 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532bb8a-82ab-4f49-8295-b48a5c635e2e admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:16.901034 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532bb8a-82ab-4f49-8295-b48a5c635e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:16.904869 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532bb8a-82ab-4f49-8295-b48a5c635e2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:16.905629 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532bb8a-82ab-4f49-8295-b48a5c635e2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:16.907811 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532bb8a-82ab-4f49-8295-b48a5c635e2e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:16.911664 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4820abc-6bb3-42bb-b230-bb2b5b66acb9 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:16.921807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4820abc-6bb3-42bb-b230-bb2b5b66acb9 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:16.927995 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5698591-073d-4f12-82ce-fd82e35c71f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:16.928722 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5698591-073d-4f12-82ce-fd82e35c71f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:16.929078 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5698591-073d-4f12-82ce-fd82e35c71f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:16.929356 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5698591-073d-4f12-82ce-fd82e35c71f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:16.929996 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5698591-073d-4f12-82ce-fd82e35c71f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:16.932396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5698591-073d-4f12-82ce-fd82e35c71f2 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:16.937571 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 743/1499] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:16 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:05:16.942519 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532bb8a-82ab-4f49-8295-b48a5c635e2e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:16.963651 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 757/1500] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:16 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:05:16.967920 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5698591-073d-4f12-82ce-fd82e35c71f2 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:17.010898 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532bb8a-82ab-4f49-8295-b48a5c635e2e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:17.030453 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:17.035240 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5698591-073d-4f12-82ce-fd82e35c71f2 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:17.039967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d0e10-e0ee-442d-b1c4-c92477409cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:17.040747 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d0e10-e0ee-442d-b1c4-c92477409cb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:17.041030 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d0e10-e0ee-442d-b1c4-c92477409cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:17.041299 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d0e10-e0ee-442d-b1c4-c92477409cb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:17.041926 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1d0e10-e0ee-442d-b1c4-c92477409cb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:17.043864 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1d0e10-e0ee-442d-b1c4-c92477409cb3 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:17.069595 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 758/1501] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:16 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:17.073457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d0e10-e0ee-442d-b1c4-c92477409cb3 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:17.081426 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c3f5e-05ef-41fc-91dc-dc598e378be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:17.082144 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c3f5e-05ef-41fc-91dc-dc598e378be1 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:17.082387 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c3f5e-05ef-41fc-91dc-dc598e378be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:17.082597 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c3f5e-05ef-41fc-91dc-dc598e378be1 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:17.083224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c3f5e-05ef-41fc-91dc-dc598e378be1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:17.084444 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d0e10-e0ee-442d-b1c4-c92477409cb3 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:17.085258 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c3f5e-05ef-41fc-91dc-dc598e378be1 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:17.089187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26ab25-5cf4-4e18-bd79-94490ba02991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:17.089821 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26ab25-5cf4-4e18-bd79-94490ba02991 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:17.090095 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26ab25-5cf4-4e18-bd79-94490ba02991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:17.090358 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26ab25-5cf4-4e18-bd79-94490ba02991 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:17.090989 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26ab25-5cf4-4e18-bd79-94490ba02991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:17.092725 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26ab25-5cf4-4e18-bd79-94490ba02991 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:17.105797 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 759/1502] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:05:17.110565 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26ab25-5cf4-4e18-bd79-94490ba02991 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:17.115852 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 744/1503] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:16 2022] POST /identity/v3/users => generated 288 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:05:17.119681 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c3f5e-05ef-41fc-91dc-dc598e378be1 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:17.123186 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26ab25-5cf4-4e18-bd79-94490ba02991 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:17.129228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff337819-5c60-41f8-b748-030ef6770194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:17.129911 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff337819-5c60-41f8-b748-030ef6770194 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:17.130192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff337819-5c60-41f8-b748-030ef6770194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:17.130291 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c3f5e-05ef-41fc-91dc-dc598e378be1 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:17.130488 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff337819-5c60-41f8-b748-030ef6770194 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:17.131094 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff337819-5c60-41f8-b748-030ef6770194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:17.132711 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff337819-5c60-41f8-b748-030ef6770194 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:17.135681 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779b67b-7851-45a6-ba8f-733afbf610ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:17.136285 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779b67b-7851-45a6-ba8f-733afbf610ca admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:17.136559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779b67b-7851-45a6-ba8f-733afbf610ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:17.136856 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779b67b-7851-45a6-ba8f-733afbf610ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:17.137486 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f779b67b-7851-45a6-ba8f-733afbf610ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:17.139060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f779b67b-7851-45a6-ba8f-733afbf610ca admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:17.144356 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 760/1504] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:17.149159 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff337819-5c60-41f8-b748-030ef6770194 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:17.155750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 745/1505] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:17.157389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779b67b-7851-45a6-ba8f-733afbf610ca admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:17.160612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff337819-5c60-41f8-b748-030ef6770194 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:17.165887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9e547-eb80-4dff-bb99-17dda2e75995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:17.166642 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9e547-eb80-4dff-bb99-17dda2e75995 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:17.166923 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9e547-eb80-4dff-bb99-17dda2e75995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:17.167199 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9e547-eb80-4dff-bb99-17dda2e75995 admin admin] PATH_INFO: `/v3/projects/b448df9b9fc5400bbd619277ab182ab9/users/1cdb02c4902b4003810b164f3ee87a2e/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:17.171289 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779b67b-7851-45a6-ba8f-733afbf610ca admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:17.177994 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0e2d8-9a36-465a-be98-5e9b57f9dfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:17.178540 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0e2d8-9a36-465a-be98-5e9b57f9dfbd admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:17.178956 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0e2d8-9a36-465a-be98-5e9b57f9dfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:17.179214 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0e2d8-9a36-465a-be98-5e9b57f9dfbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:17.179854 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff0e2d8-9a36-465a-be98-5e9b57f9dfbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:17.181537 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff0e2d8-9a36-465a-be98-5e9b57f9dfbd admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:17.183604 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9e547-eb80-4dff-bb99-17dda2e75995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b448df9b9fc5400bbd619277ab182ab9, user_id=1cdb02c4902b4003810b164f3ee87a2e, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:17.186377 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9e547-eb80-4dff-bb99-17dda2e75995 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:17.194492 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 746/1506] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:17.198833 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0e2d8-9a36-465a-be98-5e9b57f9dfbd admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:17.203395 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 761/1507] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:17 2022] PUT /identity/v3/projects/b448df9b9fc5400bbd619277ab182ab9/users/1cdb02c4902b4003810b164f3ee87a2e/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:05:17.218506 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0e2d8-9a36-465a-be98-5e9b57f9dfbd admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:17.219066 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53251e2b-45db-4dca-87ab-dcc0e1587e3a None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:17.219351 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53251e2b-45db-4dca-87ab-dcc0e1587e3a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:17.219639 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53251e2b-45db-4dca-87ab-dcc0e1587e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:17.236615 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=88,keystone:DELETE=6,keystone:INSERT=6 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:17.252806 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-53251e2b-45db-4dca-87ab-dcc0e1587e3a None None] MFA Rules not processed for user `1cdb02c4902b4003810b164f3ee87a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:17.254164 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-53251e2b-45db-4dca-87ab-dcc0e1587e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:17.283984 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecdeca-e641-413b-96e8-96a1b07c7a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:17.285266 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecdeca-e641-413b-96e8-96a1b07c7a8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:17.285619 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecdeca-e641-413b-96e8-96a1b07c7a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:17.285926 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecdeca-e641-413b-96e8-96a1b07c7a8c admin admin] PATH_INFO: `/v3/projects/8378e1b787a849da80075035ae75f25a/users/b4bf01bb6046421ea05f3e40a8b36900/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:17.306491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eecdeca-e641-413b-96e8-96a1b07c7a8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8378e1b787a849da80075035ae75f25a, user_id=b4bf01bb6046421ea05f3e40a8b36900, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:17.310000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eecdeca-e641-413b-96e8-96a1b07c7a8c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:17.327789 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 747/1508] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:17 2022] PUT /identity/v3/projects/8378e1b787a849da80075035ae75f25a/users/b4bf01bb6046421ea05f3e40a8b36900/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:17.331938 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 762/1509] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:05:17.340930 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf606d-46de-4282-b056-930e350c0fc7 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:17.341172 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecdeca-e641-413b-96e8-96a1b07c7a8c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:17.341463 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf606d-46de-4282-b056-930e350c0fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:17.341870 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf606d-46de-4282-b056-930e350c0fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:17.379383 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d6bf606d-46de-4282-b056-930e350c0fc7 None None] MFA Rules not processed for user `b4bf01bb6046421ea05f3e40a8b36900`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:17.381952 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d6bf606d-46de-4282-b056-930e350c0fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:17.441734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecdeca-e641-413b-96e8-96a1b07c7a8c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:17.448026 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ba999d-572a-4b7f-a89d-71b0670c4c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:17.448798 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba999d-572a-4b7f-a89d-71b0670c4c3d admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:17.449452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba999d-572a-4b7f-a89d-71b0670c4c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:17.449779 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba999d-572a-4b7f-a89d-71b0670c4c3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:17.450516 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ba999d-572a-4b7f-a89d-71b0670c4c3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:17.453461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ba999d-572a-4b7f-a89d-71b0670c4c3d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:17.482506 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:17.497134 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 748/1510] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:17 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:05:17.500256 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ba999d-572a-4b7f-a89d-71b0670c4c3d admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:17.523836 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 763/1511] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:17.529777 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf606d-46de-4282-b056-930e350c0fc7 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:17.543218 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf606d-46de-4282-b056-930e350c0fc7 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:17.583598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ba999d-572a-4b7f-a89d-71b0670c4c3d admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:17.592415 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a962899-7388-454d-9db5-3c3ebb3a355f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:17.593192 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a962899-7388-454d-9db5-3c3ebb3a355f admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:17.593969 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a962899-7388-454d-9db5-3c3ebb3a355f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:17.594308 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a962899-7388-454d-9db5-3c3ebb3a355f admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:17.595195 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a962899-7388-454d-9db5-3c3ebb3a355f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:17.597507 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a962899-7388-454d-9db5-3c3ebb3a355f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:17.626886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 749/1512] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:17 2022] POST /identity/v3/users => generated 299 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:17.637742 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a962899-7388-454d-9db5-3c3ebb3a355f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:17.649486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331f329-bbce-4cb6-9d65-46e51767be00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:17.651035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331f329-bbce-4cb6-9d65-46e51767be00 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:17.651035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331f329-bbce-4cb6-9d65-46e51767be00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:17.651035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331f329-bbce-4cb6-9d65-46e51767be00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:17.651485 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d331f329-bbce-4cb6-9d65-46e51767be00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:17.654542 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d331f329-bbce-4cb6-9d65-46e51767be00 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:17.664885 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a962899-7388-454d-9db5-3c3ebb3a355f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:17.677108 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb3493-f915-4a50-9e57-19dd73922c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:17.677734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb3493-f915-4a50-9e57-19dd73922c3a admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:17.678009 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb3493-f915-4a50-9e57-19dd73922c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:17.678280 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb3493-f915-4a50-9e57-19dd73922c3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:17.678921 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb3493-f915-4a50-9e57-19dd73922c3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:17.680506 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb3493-f915-4a50-9e57-19dd73922c3a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:17.706723 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 764/1513] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:17 2022] POST /identity/v3/projects => generated 377 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:17.724729 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331f329-bbce-4cb6-9d65-46e51767be00 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:17.726186 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 750/1514] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:17.730417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb3493-f915-4a50-9e57-19dd73922c3a admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:17.743644 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb3493-f915-4a50-9e57-19dd73922c3a admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:17.745219 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:17.831780 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331f329-bbce-4cb6-9d65-46e51767be00 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:17.848375 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded79e7-c0e4-4458-96dd-627cb0950b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:17.852473 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded79e7-c0e4-4458-96dd-627cb0950b0b admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:17.852792 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded79e7-c0e4-4458-96dd-627cb0950b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:17.853069 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded79e7-c0e4-4458-96dd-627cb0950b0b admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:17.855423 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ded79e7-c0e4-4458-96dd-627cb0950b0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:17.878058 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ded79e7-c0e4-4458-96dd-627cb0950b0b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:17.891781 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f338972-1fe2-48a8-be70-aef61be63720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:17.897807 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f338972-1fe2-48a8-be70-aef61be63720 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:17.899473 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f338972-1fe2-48a8-be70-aef61be63720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:17.899870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f338972-1fe2-48a8-be70-aef61be63720 admin admin] PATH_INFO: `/v3/projects/7f97e0090e6246a88b13954a3ef5b8c2/users/944b80503db541bfbc190293b2433ddb/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:17.924982 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f338972-1fe2-48a8-be70-aef61be63720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f97e0090e6246a88b13954a3ef5b8c2, user_id=944b80503db541bfbc190293b2433ddb, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:17.930233 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f338972-1fe2-48a8-be70-aef61be63720 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:17.956843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 751/1515] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:17 2022] PUT /identity/v3/projects/7f97e0090e6246a88b13954a3ef5b8c2/users/944b80503db541bfbc190293b2433ddb/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:17.962601 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f338972-1fe2-48a8-be70-aef61be63720 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:17.975486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 765/1516] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:17 2022] POST /identity/v3/users => generated 289 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:17.980072 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded79e7-c0e4-4458-96dd-627cb0950b0b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:18.010196 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded79e7-c0e4-4458-96dd-627cb0950b0b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:18.118402 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d30f158-f62d-4acb-aeee-d43c29dfa84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:18.119202 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30f158-f62d-4acb-aeee-d43c29dfa84f admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:18.119546 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30f158-f62d-4acb-aeee-d43c29dfa84f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:18.119882 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30f158-f62d-4acb-aeee-d43c29dfa84f admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:18.120628 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d30f158-f62d-4acb-aeee-d43c29dfa84f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:18.121793 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f338972-1fe2-48a8-be70-aef61be63720 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:18.122748 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d30f158-f62d-4acb-aeee-d43c29dfa84f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:18.127808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdce296-a311-4599-a6d5-e193d7ccb27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:18.128627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdce296-a311-4599-a6d5-e193d7ccb27d admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:18.129274 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdce296-a311-4599-a6d5-e193d7ccb27d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:18.129274 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdce296-a311-4599-a6d5-e193d7ccb27d admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:18.129856 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdce296-a311-4599-a6d5-e193d7ccb27d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:18.131585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdce296-a311-4599-a6d5-e193d7ccb27d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:18.133095 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:18.138854 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 766/1517] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:18.144428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d30f158-f62d-4acb-aeee-d43c29dfa84f admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:18.157513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 752/1518] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:18.161117 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdce296-a311-4599-a6d5-e193d7ccb27d admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:18.162010 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d30f158-f62d-4acb-aeee-d43c29dfa84f admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:18.174202 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcba3d0-0253-4a5f-952e-c0aa4eb55b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:18.174833 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcba3d0-0253-4a5f-952e-c0aa4eb55b78 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:18.175638 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcba3d0-0253-4a5f-952e-c0aa4eb55b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:18.175920 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcba3d0-0253-4a5f-952e-c0aa4eb55b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:18.176640 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcba3d0-0253-4a5f-952e-c0aa4eb55b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:18.178433 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdce296-a311-4599-a6d5-e193d7ccb27d admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:18.180671 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcba3d0-0253-4a5f-952e-c0aa4eb55b78 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:18.191839 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd2311-13e8-40de-8cad-3f52622eac16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:18.194125 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd2311-13e8-40de-8cad-3f52622eac16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:18.194443 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd2311-13e8-40de-8cad-3f52622eac16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:18.194741 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd2311-13e8-40de-8cad-3f52622eac16 admin admin] PATH_INFO: `/v3/projects/7f97e0090e6246a88b13954a3ef5b8c2/users/944b80503db541bfbc190293b2433ddb/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:18.198917 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd2311-13e8-40de-8cad-3f52622eac16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f97e0090e6246a88b13954a3ef5b8c2, user_id=944b80503db541bfbc190293b2433ddb, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:18.203765 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd2311-13e8-40de-8cad-3f52622eac16 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:18.205246 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 767/1519] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:18.209515 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcba3d0-0253-4a5f-952e-c0aa4eb55b78 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:18.226058 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcba3d0-0253-4a5f-952e-c0aa4eb55b78 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:18.249465 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 753/1520] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:18 2022] PUT /identity/v3/projects/7f97e0090e6246a88b13954a3ef5b8c2/users/944b80503db541bfbc190293b2433ddb/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:18.263594 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b6e0e-4168-4adf-9e56-9587747cb2a9 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:18.263885 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b6e0e-4168-4adf-9e56-9587747cb2a9 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:18.264183 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b6e0e-4168-4adf-9e56-9587747cb2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:18.314687 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-b14b6e0e-4168-4adf-9e56-9587747cb2a9 None None] MFA Rules not processed for user `944b80503db541bfbc190293b2433ddb`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:18.316530 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b14b6e0e-4168-4adf-9e56-9587747cb2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:18.343623 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d2c64-4284-4f21-a049-0e13b3d36ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:18.344356 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d2c64-4284-4f21-a049-0e13b3d36ac0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:18.344638 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d2c64-4284-4f21-a049-0e13b3d36ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:18.344955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d2c64-4284-4f21-a049-0e13b3d36ac0 admin admin] PATH_INFO: `/v3/projects/d8f9b113d0bb44c6a5e4f30e6b529600/users/b320b593ffb7431086fdc069ff67a33e/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:18.368012 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735d2c64-4284-4f21-a049-0e13b3d36ac0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8f9b113d0bb44c6a5e4f30e6b529600, user_id=b320b593ffb7431086fdc069ff67a33e, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:18.373063 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735d2c64-4284-4f21-a049-0e13b3d36ac0 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:18.405471 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 768/1521] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:18 2022] PUT /identity/v3/projects/d8f9b113d0bb44c6a5e4f30e6b529600/users/b320b593ffb7431086fdc069ff67a33e/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:05:18.407633 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c84a2-9c60-44c7-8fb7-2e49c0085636 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:18.408326 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c84a2-9c60-44c7-8fb7-2e49c0085636 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:18.408717 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c84a2-9c60-44c7-8fb7-2e49c0085636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:18.433343 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:18.439261 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 754/1522] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:18 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:18.445450 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-c00c84a2-9c60-44c7-8fb7-2e49c0085636 None None] MFA Rules not processed for user `b320b593ffb7431086fdc069ff67a33e`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:18.447817 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c00c84a2-9c60-44c7-8fb7-2e49c0085636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:18.449901 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b6e0e-4168-4adf-9e56-9587747cb2a9 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:18.451534 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b14b6e0e-4168-4adf-9e56-9587747cb2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:18.519505 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 769/1523] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:18.523830 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c84a2-9c60-44c7-8fb7-2e49c0085636 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:18.537652 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b6e0e-4168-4adf-9e56-9587747cb2a9 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:18.591926 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c84a2-9c60-44c7-8fb7-2e49c0085636 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:18.594848 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd392ee-a0e2-43ae-9792-bc63467e6480 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:18.594848 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd392ee-a0e2-43ae-9792-bc63467e6480 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:18.594848 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd392ee-a0e2-43ae-9792-bc63467e6480 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:18.595304 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd392ee-a0e2-43ae-9792-bc63467e6480 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:18.597463 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176406db-17cf-4158-9b60-e2a9aa170e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:18.598166 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176406db-17cf-4158-9b60-e2a9aa170e37 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:18.598457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176406db-17cf-4158-9b60-e2a9aa170e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:18.598761 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176406db-17cf-4158-9b60-e2a9aa170e37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:18.599453 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176406db-17cf-4158-9b60-e2a9aa170e37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:18.601467 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176406db-17cf-4158-9b60-e2a9aa170e37 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:18.639665 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:18.643171 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 770/1524] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:18 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:18.645206 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176406db-17cf-4158-9b60-e2a9aa170e37 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:18.670688 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd392ee-a0e2-43ae-9792-bc63467e6480 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:18.671892 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd392ee-a0e2-43ae-9792-bc63467e6480 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:18.682273 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 755/1525] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:18 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:05:18.728476 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176406db-17cf-4158-9b60-e2a9aa170e37 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:18.734200 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3efab1-d953-4b4f-bfc2-536ef7da72aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:18.734892 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3efab1-d953-4b4f-bfc2-536ef7da72aa admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:18.735241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3efab1-d953-4b4f-bfc2-536ef7da72aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:18.735571 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3efab1-d953-4b4f-bfc2-536ef7da72aa admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:18.736271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3efab1-d953-4b4f-bfc2-536ef7da72aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:18.739220 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3efab1-d953-4b4f-bfc2-536ef7da72aa admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:18.765221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 771/1526] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:18 2022] POST /identity/v3/users => generated 295 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:18.774216 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd392ee-a0e2-43ae-9792-bc63467e6480 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:18.785956 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd392ee-a0e2-43ae-9792-bc63467e6480 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:18.790884 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c9377-2545-4069-bc60-699a8271469c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:18.791515 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c9377-2545-4069-bc60-699a8271469c admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:18.792750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c9377-2545-4069-bc60-699a8271469c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:18.792750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c9377-2545-4069-bc60-699a8271469c admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:18.793120 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c9377-2545-4069-bc60-699a8271469c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:18.794866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c9377-2545-4069-bc60-699a8271469c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:18.809839 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 756/1527] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:05:18.813837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3efab1-d953-4b4f-bfc2-536ef7da72aa admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:18.825958 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3efab1-d953-4b4f-bfc2-536ef7da72aa admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:18.834058 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3df620-8420-4e2f-8864-61f22504948f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:18.835838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3df620-8420-4e2f-8864-61f22504948f admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:18.836143 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3df620-8420-4e2f-8864-61f22504948f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:18.836417 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3df620-8420-4e2f-8864-61f22504948f admin admin] PATH_INFO: `/v3/projects/a493ad3ccaf341cda57ef1070ffd11f9/users/6d382677274e431e82b31a49bfddc382/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:18.852541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:18.858185 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3df620-8420-4e2f-8864-61f22504948f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a493ad3ccaf341cda57ef1070ffd11f9, user_id=6d382677274e431e82b31a49bfddc382, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:18.861814 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3df620-8420-4e2f-8864-61f22504948f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:18.881279 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 772/1528] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:18 2022] PUT /identity/v3/projects/a493ad3ccaf341cda57ef1070ffd11f9/users/6d382677274e431e82b31a49bfddc382/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:18.886100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c9377-2545-4069-bc60-699a8271469c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:18.904138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c9377-2545-4069-bc60-699a8271469c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:18.978502 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277576b-ff98-4e76-9d4d-5266939a3935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:18.979147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277576b-ff98-4e76-9d4d-5266939a3935 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:18.979425 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277576b-ff98-4e76-9d4d-5266939a3935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:18.979696 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277576b-ff98-4e76-9d4d-5266939a3935 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:18.980340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5277576b-ff98-4e76-9d4d-5266939a3935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:18.981952 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5277576b-ff98-4e76-9d4d-5266939a3935 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:18.995089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 757/1529] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:18.999364 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3df620-8420-4e2f-8864-61f22504948f admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:19.017741 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3df620-8420-4e2f-8864-61f22504948f admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:19.028911 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1bfb3-19af-406c-8723-be25ed073816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:19.029911 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1bfb3-19af-406c-8723-be25ed073816 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:19.030385 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1bfb3-19af-406c-8723-be25ed073816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:19.031058 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1bfb3-19af-406c-8723-be25ed073816 admin admin] PATH_INFO: `/v3/projects/a493ad3ccaf341cda57ef1070ffd11f9/users/6d382677274e431e82b31a49bfddc382/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:19.033760 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c1bfb3-19af-406c-8723-be25ed073816 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a493ad3ccaf341cda57ef1070ffd11f9, user_id=6d382677274e431e82b31a49bfddc382, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:19.038347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c1bfb3-19af-406c-8723-be25ed073816 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:19.052562 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 773/1530] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:18 2022] PUT /identity/v3/projects/a493ad3ccaf341cda57ef1070ffd11f9/users/6d382677274e431e82b31a49bfddc382/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:19.063067 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9db87-a115-4d4c-81ec-b29f61d91219 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:19.063638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9db87-a115-4d4c-81ec-b29f61d91219 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:19.063982 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9db87-a115-4d4c-81ec-b29f61d91219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:19.103427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-dcd9db87-a115-4d4c-81ec-b29f61d91219 None None] MFA Rules not processed for user `6d382677274e431e82b31a49bfddc382`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:19.105109 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-dcd9db87-a115-4d4c-81ec-b29f61d91219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:19.215820 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 758/1531] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:19.228191 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1bfb3-19af-406c-8723-be25ed073816 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:19.319689 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1bfb3-19af-406c-8723-be25ed073816 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:19.324142 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17149fdf-cafe-4367-b569-fef6f7e16e0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:19.324943 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17149fdf-cafe-4367-b569-fef6f7e16e0f service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:19.325516 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17149fdf-cafe-4367-b569-fef6f7e16e0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:19.325976 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17149fdf-cafe-4367-b569-fef6f7e16e0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:19.428036 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17149fdf-cafe-4367-b569-fef6f7e16e0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:19.429665 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17149fdf-cafe-4367-b569-fef6f7e16e0f service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:19.451257 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 774/1532] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:19.509616 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9db87-a115-4d4c-81ec-b29f61d91219 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:19.601217 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:19.654310 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9db87-a115-4d4c-81ec-b29f61d91219 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:19.665627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29e4c3-8bfd-4d6b-a665-4a593ef0b63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:19.666540 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29e4c3-8bfd-4d6b-a665-4a593ef0b63f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:19.667624 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29e4c3-8bfd-4d6b-a665-4a593ef0b63f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:19.668018 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29e4c3-8bfd-4d6b-a665-4a593ef0b63f admin admin] PATH_INFO: `/v3/users/1cdb02c4902b4003810b164f3ee87a2e` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:19.669270 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d29e4c3-8bfd-4d6b-a665-4a593ef0b63f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cdb02c4902b4003810b164f3ee87a2e)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:19.671384 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d29e4c3-8bfd-4d6b-a665-4a593ef0b63f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:19.798728 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4d29e4c3-8bfd-4d6b-a665-4a593ef0b63f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cdb02c4902b4003810b164f3ee87a2e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:19.799541 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4d29e4c3-8bfd-4d6b-a665-4a593ef0b63f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cdb02c4902b4003810b164f3ee87a2e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:19.833865 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4d29e4c3-8bfd-4d6b-a665-4a593ef0b63f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cdb02c4902b4003810b164f3ee87a2e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:19.863380 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4d29e4c3-8bfd-4d6b-a665-4a593ef0b63f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cdb02c4902b4003810b164f3ee87a2e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:19.885597 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4d29e4c3-8bfd-4d6b-a665-4a593ef0b63f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cdb02c4902b4003810b164f3ee87a2e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:19.893461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 759/1533] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:19 2022] DELETE /identity/v3/users/1cdb02c4902b4003810b164f3ee87a2e => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:05:19.899701 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42faa7c2-65b2-4d81-be1b-7421c90aaff7 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:19.900053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42faa7c2-65b2-4d81-be1b-7421c90aaff7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:19.900307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42faa7c2-65b2-4d81-be1b-7421c90aaff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:19.937822 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-42faa7c2-65b2-4d81-be1b-7421c90aaff7 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:19.942184 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-42faa7c2-65b2-4d81-be1b-7421c90aaff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:20.038720 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 775/1534] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:20.048386 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29e4c3-8bfd-4d6b-a665-4a593ef0b63f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:20.049878 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-4d29e4c3-8bfd-4d6b-a665-4a593ef0b63f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:20.118145 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29e4c3-8bfd-4d6b-a665-4a593ef0b63f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:20.124318 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa28add-e4e5-45a7-98f8-f7a6d14f7419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:20.125529 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa28add-e4e5-45a7-98f8-f7a6d14f7419 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:20.126174 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa28add-e4e5-45a7-98f8-f7a6d14f7419 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:20.126583 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa28add-e4e5-45a7-98f8-f7a6d14f7419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:20.139466 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa28add-e4e5-45a7-98f8-f7a6d14f7419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:20.141262 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa28add-e4e5-45a7-98f8-f7a6d14f7419 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:20.153169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 760/1535] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:20.217869 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42faa7c2-65b2-4d81-be1b-7421c90aaff7 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:20.219339 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-42faa7c2-65b2-4d81-be1b-7421c90aaff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:20.234675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42faa7c2-65b2-4d81-be1b-7421c90aaff7 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:20.244026 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac1081-c86c-48ef-a6be-bcfd0989b022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:20.245062 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac1081-c86c-48ef-a6be-bcfd0989b022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:20.245673 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac1081-c86c-48ef-a6be-bcfd0989b022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:20.246153 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac1081-c86c-48ef-a6be-bcfd0989b022 admin admin] PATH_INFO: `/v3/users/f8b6fcf434264a7c8f98840220ec4b03` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:20.247824 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac1081-c86c-48ef-a6be-bcfd0989b022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8b6fcf434264a7c8f98840220ec4b03)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:20.251438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac1081-c86c-48ef-a6be-bcfd0989b022 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:20.293197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:20.335523 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f0ac1081-c86c-48ef-a6be-bcfd0989b022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8b6fcf434264a7c8f98840220ec4b03'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:20.336339 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f0ac1081-c86c-48ef-a6be-bcfd0989b022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8b6fcf434264a7c8f98840220ec4b03'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:20.349568 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f0ac1081-c86c-48ef-a6be-bcfd0989b022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8b6fcf434264a7c8f98840220ec4b03'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:20.386987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f0ac1081-c86c-48ef-a6be-bcfd0989b022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8b6fcf434264a7c8f98840220ec4b03'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:20.387721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f0ac1081-c86c-48ef-a6be-bcfd0989b022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8b6fcf434264a7c8f98840220ec4b03'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:20.402962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 776/1536] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:20 2022] DELETE /identity/v3/users/f8b6fcf434264a7c8f98840220ec4b03 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:20.407439 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e3a11-bbea-4254-b025-a1989d505eac None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:20.407730 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e3a11-bbea-4254-b025-a1989d505eac None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:20.408011 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e3a11-bbea-4254-b025-a1989d505eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:20.432900 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-965e3a11-bbea-4254-b025-a1989d505eac None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:20.434720 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-965e3a11-bbea-4254-b025-a1989d505eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:20.519524 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 761/1537] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:20.535891 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac1081-c86c-48ef-a6be-bcfd0989b022 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:20.538426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f0ac1081-c86c-48ef-a6be-bcfd0989b022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:20.571340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e3a11-bbea-4254-b025-a1989d505eac None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:20.575242 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-965e3a11-bbea-4254-b025-a1989d505eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:20.605883 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e3a11-bbea-4254-b025-a1989d505eac None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:20.616799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1296f8d9-0c1d-40a3-bad3-6aa08976713d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:20.617322 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296f8d9-0c1d-40a3-bad3-6aa08976713d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:20.617605 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296f8d9-0c1d-40a3-bad3-6aa08976713d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:20.617874 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296f8d9-0c1d-40a3-bad3-6aa08976713d admin admin] PATH_INFO: `/v3/projects/b448df9b9fc5400bbd619277ab182ab9` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:20.622019 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1296f8d9-0c1d-40a3-bad3-6aa08976713d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b448df9b9fc5400bbd619277ab182ab9)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:20.623956 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1296f8d9-0c1d-40a3-bad3-6aa08976713d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:20.626259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:20.683417 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac1081-c86c-48ef-a6be-bcfd0989b022 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:20.690321 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8b226-d76e-4906-8773-9b94ec4272b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:20.692219 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8b226-d76e-4906-8773-9b94ec4272b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:20.692506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8b226-d76e-4906-8773-9b94ec4272b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:20.692833 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8b226-d76e-4906-8773-9b94ec4272b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:20.723848 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b8b226-d76e-4906-8773-9b94ec4272b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:20.725578 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b8b226-d76e-4906-8773-9b94ec4272b4 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:20.726635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1296f8d9-0c1d-40a3-bad3-6aa08976713d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b448df9b9fc5400bbd619277ab182ab9', 'request_id': 'req-1296f8d9-0c1d-40a3-bad3-6aa08976713d'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:20.734702 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1296f8d9-0c1d-40a3-bad3-6aa08976713d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b448df9b9fc5400bbd619277ab182ab9', 'request_id': 'req-1296f8d9-0c1d-40a3-bad3-6aa08976713d'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:20.759288 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 777/1538] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:20.760394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1296f8d9-0c1d-40a3-bad3-6aa08976713d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b448df9b9fc5400bbd619277ab182ab9', 'request_id': 'req-1296f8d9-0c1d-40a3-bad3-6aa08976713d'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:20.801594 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1296f8d9-0c1d-40a3-bad3-6aa08976713d admin admin] The token cache is being invalidate because project b448df9b9fc5400bbd619277ab182ab9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:20.802017 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1296f8d9-0c1d-40a3-bad3-6aa08976713d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:20.802854 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1296f8d9-0c1d-40a3-bad3-6aa08976713d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:20.805594 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 762/1539] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:20 2022] DELETE /identity/v3/projects/b448df9b9fc5400bbd619277ab182ab9 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:05:20.810600 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8b226-d76e-4906-8773-9b94ec4272b4 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:20.821576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8b226-d76e-4906-8773-9b94ec4272b4 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:20.918079 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f2ad8-6886-44f9-9f69-09aa5d1d0019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:20.918940 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f2ad8-6886-44f9-9f69-09aa5d1d0019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:20.919329 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f2ad8-6886-44f9-9f69-09aa5d1d0019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:20.920305 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f2ad8-6886-44f9-9f69-09aa5d1d0019 admin admin] PATH_INFO: `/v3/users/944b80503db541bfbc190293b2433ddb` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:20.922699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f2ad8-6886-44f9-9f69-09aa5d1d0019 admin admin] RBAC: Authorizing `identity:delete_user(user_id=944b80503db541bfbc190293b2433ddb)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:20.928379 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f2ad8-6886-44f9-9f69-09aa5d1d0019 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:21.021230 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-453f2ad8-6886-44f9-9f69-09aa5d1d0019 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '944b80503db541bfbc190293b2433ddb'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:21.021967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-453f2ad8-6886-44f9-9f69-09aa5d1d0019 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '944b80503db541bfbc190293b2433ddb'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:21.024943 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:21.036106 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-453f2ad8-6886-44f9-9f69-09aa5d1d0019 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '944b80503db541bfbc190293b2433ddb'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:21.053946 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-453f2ad8-6886-44f9-9f69-09aa5d1d0019 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '944b80503db541bfbc190293b2433ddb'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:21.054655 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-453f2ad8-6886-44f9-9f69-09aa5d1d0019 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '944b80503db541bfbc190293b2433ddb'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:21.071921 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 778/1540] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:20 2022] DELETE /identity/v3/users/944b80503db541bfbc190293b2433ddb => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:21.213648 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1296f8d9-0c1d-40a3-bad3-6aa08976713d admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:21.215175 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1296f8d9-0c1d-40a3-bad3-6aa08976713d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:21.287888 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1296f8d9-0c1d-40a3-bad3-6aa08976713d admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:21.353809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3b458-712f-46d4-8487-cf43ef8a82a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:21.354418 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3b458-712f-46d4-8487-cf43ef8a82a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:21.354695 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3b458-712f-46d4-8487-cf43ef8a82a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:21.354966 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3b458-712f-46d4-8487-cf43ef8a82a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:21.357185 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d2e3b458-712f-46d4-8487-cf43ef8a82a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:21.428542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3b458-712f-46d4-8487-cf43ef8a82a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:21.430052 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3b458-712f-46d4-8487-cf43ef8a82a2 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:21.435758 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:21.443545 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 763/1541] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:21.918684 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f2ad8-6886-44f9-9f69-09aa5d1d0019 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:21.920677 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-453f2ad8-6886-44f9-9f69-09aa5d1d0019 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:22.052539 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f2ad8-6886-44f9-9f69-09aa5d1d0019 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:22.060406 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735607a-d194-41f6-9fa8-7b636bfecd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:22.061438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735607a-d194-41f6-9fa8-7b636bfecd74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:22.062047 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735607a-d194-41f6-9fa8-7b636bfecd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:22.062557 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735607a-d194-41f6-9fa8-7b636bfecd74 admin admin] PATH_INFO: `/v3/projects/7a0d68c2055e47eaaca5f05313aff446` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:22.064275 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735607a-d194-41f6-9fa8-7b636bfecd74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a0d68c2055e47eaaca5f05313aff446)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:22.067261 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735607a-d194-41f6-9fa8-7b636bfecd74 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:22.180029 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a735607a-d194-41f6-9fa8-7b636bfecd74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a0d68c2055e47eaaca5f05313aff446', 'request_id': 'req-a735607a-d194-41f6-9fa8-7b636bfecd74'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:22.210582 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a735607a-d194-41f6-9fa8-7b636bfecd74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a0d68c2055e47eaaca5f05313aff446', 'request_id': 'req-a735607a-d194-41f6-9fa8-7b636bfecd74'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:22.225964 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a735607a-d194-41f6-9fa8-7b636bfecd74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a0d68c2055e47eaaca5f05313aff446', 'request_id': 'req-a735607a-d194-41f6-9fa8-7b636bfecd74'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:22.265013 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a735607a-d194-41f6-9fa8-7b636bfecd74 admin admin] The token cache is being invalidate because project 7a0d68c2055e47eaaca5f05313aff446 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:22.265360 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a735607a-d194-41f6-9fa8-7b636bfecd74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:22.266467 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a735607a-d194-41f6-9fa8-7b636bfecd74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:22.268147 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 779/1542] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:21 2022] DELETE /identity/v3/projects/7a0d68c2055e47eaaca5f05313aff446 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:22.273951 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3b458-712f-46d4-8487-cf43ef8a82a2 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:22.321334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735607a-d194-41f6-9fa8-7b636bfecd74 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:22.322756 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a735607a-d194-41f6-9fa8-7b636bfecd74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:22.325118 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3b458-712f-46d4-8487-cf43ef8a82a2 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:22.496072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288aaab-739d-44fc-ac03-4e0c3eeddcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:22.497012 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:22.498892 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288aaab-739d-44fc-ac03-4e0c3eeddcce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:22.499280 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288aaab-739d-44fc-ac03-4e0c3eeddcce admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:22.499668 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288aaab-739d-44fc-ac03-4e0c3eeddcce admin admin] PATH_INFO: `/v3/users/ca89311f940b42ac9104cbc6f1e52f7f` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:22.502478 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9288aaab-739d-44fc-ac03-4e0c3eeddcce admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca89311f940b42ac9104cbc6f1e52f7f)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:22.507319 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9288aaab-739d-44fc-ac03-4e0c3eeddcce admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:22.513415 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735607a-d194-41f6-9fa8-7b636bfecd74 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:22.525551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79c59f-1918-4783-bc38-bf369fb3b9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:22.526040 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79c59f-1918-4783-bc38-bf369fb3b9b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:22.526322 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79c59f-1918-4783-bc38-bf369fb3b9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:22.526595 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79c59f-1918-4783-bc38-bf369fb3b9b5 admin admin] PATH_INFO: `/v3/projects/7f97e0090e6246a88b13954a3ef5b8c2` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:22.532176 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79c59f-1918-4783-bc38-bf369fb3b9b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f97e0090e6246a88b13954a3ef5b8c2)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:22.534955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79c59f-1918-4783-bc38-bf369fb3b9b5 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:22.597928 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9288aaab-739d-44fc-ac03-4e0c3eeddcce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca89311f940b42ac9104cbc6f1e52f7f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:22.599873 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9288aaab-739d-44fc-ac03-4e0c3eeddcce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca89311f940b42ac9104cbc6f1e52f7f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:22.615871 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9288aaab-739d-44fc-ac03-4e0c3eeddcce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca89311f940b42ac9104cbc6f1e52f7f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:22.634476 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9288aaab-739d-44fc-ac03-4e0c3eeddcce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca89311f940b42ac9104cbc6f1e52f7f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:22.644525 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9288aaab-739d-44fc-ac03-4e0c3eeddcce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca89311f940b42ac9104cbc6f1e52f7f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:22.648441 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2d79c59f-1918-4783-bc38-bf369fb3b9b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f97e0090e6246a88b13954a3ef5b8c2', 'request_id': 'req-2d79c59f-1918-4783-bc38-bf369fb3b9b5'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:22.653296 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 764/1543] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:22 2022] DELETE /identity/v3/users/ca89311f940b42ac9104cbc6f1e52f7f => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:22.654235 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288aaab-739d-44fc-ac03-4e0c3eeddcce admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:22.655692 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-9288aaab-739d-44fc-ac03-4e0c3eeddcce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:22.659479 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2d79c59f-1918-4783-bc38-bf369fb3b9b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f97e0090e6246a88b13954a3ef5b8c2', 'request_id': 'req-2d79c59f-1918-4783-bc38-bf369fb3b9b5'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:22.668758 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2d79c59f-1918-4783-bc38-bf369fb3b9b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f97e0090e6246a88b13954a3ef5b8c2', 'request_id': 'req-2d79c59f-1918-4783-bc38-bf369fb3b9b5'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:22.692711 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2d79c59f-1918-4783-bc38-bf369fb3b9b5 admin admin] The token cache is being invalidate because project 7f97e0090e6246a88b13954a3ef5b8c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:22.693062 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2d79c59f-1918-4783-bc38-bf369fb3b9b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:22.694016 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2d79c59f-1918-4783-bc38-bf369fb3b9b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:22.697741 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 780/1544] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:22 2022] DELETE /identity/v3/projects/7f97e0090e6246a88b13954a3ef5b8c2 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:22.703836 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa258c-f1a3-4537-aa33-d71404932008 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:22.704124 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa258c-f1a3-4537-aa33-d71404932008 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:22.704402 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa258c-f1a3-4537-aa33-d71404932008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:22.733125 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-b6fa258c-f1a3-4537-aa33-d71404932008 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:22.734854 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b6fa258c-f1a3-4537-aa33-d71404932008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:22.756970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288aaab-739d-44fc-ac03-4e0c3eeddcce admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:22.766326 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1deaa7-bad0-4d08-bf19-4fb9b2ae01de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:22.770845 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1deaa7-bad0-4d08-bf19-4fb9b2ae01de service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:22.771129 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1deaa7-bad0-4d08-bf19-4fb9b2ae01de service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:22.771399 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1deaa7-bad0-4d08-bf19-4fb9b2ae01de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:22.774260 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6c1deaa7-bad0-4d08-bf19-4fb9b2ae01de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:22.849401 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:22.886809 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 781/1545] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:22.889493 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea270812-987e-492c-9cfe-a66b8eaca7d8 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:22.890034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea270812-987e-492c-9cfe-a66b8eaca7d8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:22.890150 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea270812-987e-492c-9cfe-a66b8eaca7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:22.935004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1deaa7-bad0-4d08-bf19-4fb9b2ae01de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:22.936478 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1deaa7-bad0-4d08-bf19-4fb9b2ae01de service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:22.937096 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-ea270812-987e-492c-9cfe-a66b8eaca7d8 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:22.938317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ea270812-987e-492c-9cfe-a66b8eaca7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:22.958253 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 765/1546] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:22 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:22.959659 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 782/1547] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:22.963782 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1deaa7-bad0-4d08-bf19-4fb9b2ae01de service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:22.981200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1deaa7-bad0-4d08-bf19-4fb9b2ae01de service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:22.986727 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7626b-0a98-4bd0-ad7c-00ffbec027ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:22.987425 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7626b-0a98-4bd0-ad7c-00ffbec027ad admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:22.987704 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7626b-0a98-4bd0-ad7c-00ffbec027ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:22.988190 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7626b-0a98-4bd0-ad7c-00ffbec027ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:22.989629 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7626b-0a98-4bd0-ad7c-00ffbec027ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:05:22.989934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7626b-0a98-4bd0-ad7c-00ffbec027ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:22.991550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7626b-0a98-4bd0-ad7c-00ffbec027ad admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:23.008215 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 766/1548] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 06:05:23.013085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea270812-987e-492c-9cfe-a66b8eaca7d8 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:23.031226 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea270812-987e-492c-9cfe-a66b8eaca7d8 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:23.044219 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c1117-f758-414a-bdd4-258c1dc089a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:23.045078 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c1117-f758-414a-bdd4-258c1dc089a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:23.045407 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c1117-f758-414a-bdd4-258c1dc089a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:23.045713 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c1117-f758-414a-bdd4-258c1dc089a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:23.046624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c1117-f758-414a-bdd4-258c1dc089a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:23.048982 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c1117-f758-414a-bdd4-258c1dc089a9 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:23.103681 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 783/1549] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:23 2022] POST /identity/v3/projects => generated 383 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:23.107496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7626b-0a98-4bd0-ad7c-00ffbec027ad admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:23.121770 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7626b-0a98-4bd0-ad7c-00ffbec027ad admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:23.227804 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338197ac-588f-459b-9d8c-d8fb08d5e778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:23.229797 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338197ac-588f-459b-9d8c-d8fb08d5e778 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:23.230559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338197ac-588f-459b-9d8c-d8fb08d5e778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:23.231081 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338197ac-588f-459b-9d8c-d8fb08d5e778 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:23.232124 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338197ac-588f-459b-9d8c-d8fb08d5e778 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:23.234931 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338197ac-588f-459b-9d8c-d8fb08d5e778 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:23.277205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 767/1550] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:23 2022] POST /identity/v3/users => generated 292 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:23.277749 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c1117-f758-414a-bdd4-258c1dc089a9 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:23.291141 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c1117-f758-414a-bdd4-258c1dc089a9 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:23.302299 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32862c2-2a80-494d-b1c6-d3d312feaa9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:23.302299 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32862c2-2a80-494d-b1c6-d3d312feaa9f admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:23.302299 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32862c2-2a80-494d-b1c6-d3d312feaa9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:23.302299 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32862c2-2a80-494d-b1c6-d3d312feaa9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:23.302727 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32862c2-2a80-494d-b1c6-d3d312feaa9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:23.302727 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32862c2-2a80-494d-b1c6-d3d312feaa9f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:23.320000 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 784/1551] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:23.324913 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338197ac-588f-459b-9d8c-d8fb08d5e778 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:23.343128 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338197ac-588f-459b-9d8c-d8fb08d5e778 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:23.355064 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd54396a-d50b-4724-ae43-c67b92d06141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:23.355970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd54396a-d50b-4724-ae43-c67b92d06141 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:23.356437 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd54396a-d50b-4724-ae43-c67b92d06141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:23.356822 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd54396a-d50b-4724-ae43-c67b92d06141 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:23.357535 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd54396a-d50b-4724-ae43-c67b92d06141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:23.359193 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd54396a-d50b-4724-ae43-c67b92d06141 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:23.375476 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 768/1552] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:23.376633 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32862c2-2a80-494d-b1c6-d3d312feaa9f admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:23.387063 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32862c2-2a80-494d-b1c6-d3d312feaa9f admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:23.392026 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7424d-ac16-4ade-be2d-cb3837c53132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:23.392685 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7424d-ac16-4ade-be2d-cb3837c53132 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:23.392685 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7424d-ac16-4ade-be2d-cb3837c53132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:23.393377 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7424d-ac16-4ade-be2d-cb3837c53132 admin admin] PATH_INFO: `/v3/projects/a572c375448a4037aadfb45f679ea2fb/users/3d61d267f1024a77a294d3031db7e7c4/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:23.408926 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7424d-ac16-4ade-be2d-cb3837c53132 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a572c375448a4037aadfb45f679ea2fb, user_id=3d61d267f1024a77a294d3031db7e7c4, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:23.411667 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7424d-ac16-4ade-be2d-cb3837c53132 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:23.426633 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 785/1553] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:23 2022] PUT /identity/v3/projects/a572c375448a4037aadfb45f679ea2fb/users/3d61d267f1024a77a294d3031db7e7c4/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:23.437652 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfd806-35c6-429b-bf27-5ccba0459ac0 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:23.437949 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfd806-35c6-429b-bf27-5ccba0459ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:23.438205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfd806-35c6-429b-bf27-5ccba0459ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:23.467446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-82cfd806-35c6-429b-bf27-5ccba0459ac0 None None] MFA Rules not processed for user `3d61d267f1024a77a294d3031db7e7c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:23.468644 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-82cfd806-35c6-429b-bf27-5ccba0459ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:23.523892 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7424d-ac16-4ade-be2d-cb3837c53132 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:23.525997 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c1b7424d-ac16-4ade-be2d-cb3837c53132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:23.562595 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 769/1554] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:23.567507 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cfd806-35c6-429b-bf27-5ccba0459ac0 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:23.570074 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:23.617742 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:23.644879 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7424d-ac16-4ade-be2d-cb3837c53132 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:23.655431 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faceeeb-ae7f-42f1-9fbd-b84655acd002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:23.656089 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faceeeb-ae7f-42f1-9fbd-b84655acd002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:23.656392 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faceeeb-ae7f-42f1-9fbd-b84655acd002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:23.656918 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faceeeb-ae7f-42f1-9fbd-b84655acd002 admin admin] PATH_INFO: `/v3/projects/0a8a9d59285e473fba9b05d851917eda` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:23.662591 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faceeeb-ae7f-42f1-9fbd-b84655acd002 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a8a9d59285e473fba9b05d851917eda)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:23.664526 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faceeeb-ae7f-42f1-9fbd-b84655acd002 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:23.676461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cfd806-35c6-429b-bf27-5ccba0459ac0 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:23.681916 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f887f02-0e85-4b0a-97cb-67993eeee76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:23.682540 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f887f02-0e85-4b0a-97cb-67993eeee76e admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:23.682820 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f887f02-0e85-4b0a-97cb-67993eeee76e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:23.683094 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f887f02-0e85-4b0a-97cb-67993eeee76e admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:23.683728 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f887f02-0e85-4b0a-97cb-67993eeee76e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:23.685683 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f887f02-0e85-4b0a-97cb-67993eeee76e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:23.722548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 770/1555] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:23 2022] POST /identity/v3/projects => generated 381 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:05:23.725518 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5faceeeb-ae7f-42f1-9fbd-b84655acd002 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a8a9d59285e473fba9b05d851917eda', 'request_id': 'req-5faceeeb-ae7f-42f1-9fbd-b84655acd002'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:23.732041 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5faceeeb-ae7f-42f1-9fbd-b84655acd002 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a8a9d59285e473fba9b05d851917eda', 'request_id': 'req-5faceeeb-ae7f-42f1-9fbd-b84655acd002'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:23.735652 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f887f02-0e85-4b0a-97cb-67993eeee76e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:23.741554 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5faceeeb-ae7f-42f1-9fbd-b84655acd002 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a8a9d59285e473fba9b05d851917eda', 'request_id': 'req-5faceeeb-ae7f-42f1-9fbd-b84655acd002'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:23.761267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5faceeeb-ae7f-42f1-9fbd-b84655acd002 admin admin] The token cache is being invalidate because project 0a8a9d59285e473fba9b05d851917eda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:23.761612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5faceeeb-ae7f-42f1-9fbd-b84655acd002 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:23.762818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5faceeeb-ae7f-42f1-9fbd-b84655acd002 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:23.766222 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 786/1556] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:23 2022] DELETE /identity/v3/projects/0a8a9d59285e473fba9b05d851917eda => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:05:23.771133 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96335b1-d6df-48f5-a220-e6b8ce6f2814 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:23.771420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96335b1-d6df-48f5-a220-e6b8ce6f2814 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:23.771699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96335b1-d6df-48f5-a220-e6b8ce6f2814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:23.798877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-b96335b1-d6df-48f5-a220-e6b8ce6f2814 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:23.801027 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b96335b1-d6df-48f5-a220-e6b8ce6f2814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:23.811922 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 787/1557] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:05:23.813190 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f887f02-0e85-4b0a-97cb-67993eeee76e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:23.822309 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec9a15-49a1-4261-be38-9282738b7008 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:23.823223 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec9a15-49a1-4261-be38-9282738b7008 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:23.823899 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec9a15-49a1-4261-be38-9282738b7008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:23.825417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2770213-6dd6-48cc-a488-a03b08cb3406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:23.826048 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2770213-6dd6-48cc-a488-a03b08cb3406 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:23.826330 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2770213-6dd6-48cc-a488-a03b08cb3406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:23.826606 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2770213-6dd6-48cc-a488-a03b08cb3406 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:23.827247 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2770213-6dd6-48cc-a488-a03b08cb3406 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:23.830038 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2770213-6dd6-48cc-a488-a03b08cb3406 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:23.853052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-1bec9a15-49a1-4261-be38-9282738b7008 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:23.854314 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1bec9a15-49a1-4261-be38-9282738b7008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:23.865451 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 788/1558] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:23.865932 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 771/1559] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:23 2022] POST /identity/v3/users => generated 291 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:23.869730 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec9a15-49a1-4261-be38-9282738b7008 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:23.870657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2770213-6dd6-48cc-a488-a03b08cb3406 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:23.872415 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e2770213-6dd6-48cc-a488-a03b08cb3406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:23.881223 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec9a15-49a1-4261-be38-9282738b7008 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:23.886183 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2770213-6dd6-48cc-a488-a03b08cb3406 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:23.886558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e064224-67d8-4bf0-a0bc-dea0a0e2feff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:23.887211 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e064224-67d8-4bf0-a0bc-dea0a0e2feff admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:23.887440 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e064224-67d8-4bf0-a0bc-dea0a0e2feff admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:23.887709 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e064224-67d8-4bf0-a0bc-dea0a0e2feff admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:23.888345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e064224-67d8-4bf0-a0bc-dea0a0e2feff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:05:23.888940 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e064224-67d8-4bf0-a0bc-dea0a0e2feff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:23.890531 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e064224-67d8-4bf0-a0bc-dea0a0e2feff admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:23.891706 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bada7-a4e6-4280-9481-973de8b05046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:23.892390 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bada7-a4e6-4280-9481-973de8b05046 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:23.892781 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bada7-a4e6-4280-9481-973de8b05046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:23.893133 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bada7-a4e6-4280-9481-973de8b05046 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:23.894257 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07bada7-a4e6-4280-9481-973de8b05046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:23.895926 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07bada7-a4e6-4280-9481-973de8b05046 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:23.903744 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 789/1560] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:05:23.907757 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e064224-67d8-4bf0-a0bc-dea0a0e2feff admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:23.912448 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 772/1561] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:23.913658 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bada7-a4e6-4280-9481-973de8b05046 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:23.919721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e064224-67d8-4bf0-a0bc-dea0a0e2feff admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:23.924606 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bada7-a4e6-4280-9481-973de8b05046 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:23.925205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e66862-0040-4228-a9f6-572e88a73b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:23.925891 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e66862-0040-4228-a9f6-572e88a73b07 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:23.926165 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e66862-0040-4228-a9f6-572e88a73b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:23.926437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e66862-0040-4228-a9f6-572e88a73b07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:23.927284 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e66862-0040-4228-a9f6-572e88a73b07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:23.929218 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e66862-0040-4228-a9f6-572e88a73b07 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:23.930710 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d143314-8c54-4de0-aa8d-9a15531783c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:23.931362 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d143314-8c54-4de0-aa8d-9a15531783c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:23.931681 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d143314-8c54-4de0-aa8d-9a15531783c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:23.931943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d143314-8c54-4de0-aa8d-9a15531783c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:23.932606 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d143314-8c54-4de0-aa8d-9a15531783c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:23.934672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d143314-8c54-4de0-aa8d-9a15531783c3 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:23.951396 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 773/1562] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:05:23.952111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d143314-8c54-4de0-aa8d-9a15531783c3 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:23.967296 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d143314-8c54-4de0-aa8d-9a15531783c3 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:23.974728 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440a09a-588b-4803-b26a-b9ecf0193438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:23.975471 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440a09a-588b-4803-b26a-b9ecf0193438 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:23.975825 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440a09a-588b-4803-b26a-b9ecf0193438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:23.976172 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440a09a-588b-4803-b26a-b9ecf0193438 admin admin] PATH_INFO: `/v3/projects/eaba857bf28e465289cdb68ec1cb2cc5/users/70c3d673547c4f45945a97eb4921bac2/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:23.990978 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 790/1563] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:23 2022] POST /identity/v3/projects => generated 365 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:05:23.994111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e66862-0040-4228-a9f6-572e88a73b07 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:23.998943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440a09a-588b-4803-b26a-b9ecf0193438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaba857bf28e465289cdb68ec1cb2cc5, user_id=70c3d673547c4f45945a97eb4921bac2, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:24.004105 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440a09a-588b-4803-b26a-b9ecf0193438 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:24.028008 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 774/1564] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:23 2022] PUT /identity/v3/projects/eaba857bf28e465289cdb68ec1cb2cc5/users/70c3d673547c4f45945a97eb4921bac2/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:24.035286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35195d8e-fbff-4271-b62a-c48b102759c2 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:24.035568 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35195d8e-fbff-4271-b62a-c48b102759c2 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:24.035842 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35195d8e-fbff-4271-b62a-c48b102759c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:24.073640 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-35195d8e-fbff-4271-b62a-c48b102759c2 None None] MFA Rules not processed for user `70c3d673547c4f45945a97eb4921bac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:24.075630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-35195d8e-fbff-4271-b62a-c48b102759c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:24.092685 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e66862-0040-4228-a9f6-572e88a73b07 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:24.106150 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:24.136456 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:24.176758 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7c42d-b4ed-43f7-b571-7ecf6af3b341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:24.177319 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 775/1565] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:05:24.177958 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7c42d-b4ed-43f7-b571-7ecf6af3b341 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:24.178313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7c42d-b4ed-43f7-b571-7ecf6af3b341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:24.178576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7c42d-b4ed-43f7-b571-7ecf6af3b341 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:24.179209 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7c42d-b4ed-43f7-b571-7ecf6af3b341 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:24.181431 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7c42d-b4ed-43f7-b571-7ecf6af3b341 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:24.182050 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35195d8e-fbff-4271-b62a-c48b102759c2 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:24.196544 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35195d8e-fbff-4271-b62a-c48b102759c2 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:24.201957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c87d0-6488-46c8-97a6-9b4be8eb8010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:24.202736 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c87d0-6488-46c8-97a6-9b4be8eb8010 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:24.203115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c87d0-6488-46c8-97a6-9b4be8eb8010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:24.203798 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c87d0-6488-46c8-97a6-9b4be8eb8010 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:24.204267 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c87d0-6488-46c8-97a6-9b4be8eb8010 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:24.206693 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c87d0-6488-46c8-97a6-9b4be8eb8010 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:24.210098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 791/1566] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:23 2022] POST /identity/v3/users => generated 283 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:24.213899 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7c42d-b4ed-43f7-b571-7ecf6af3b341 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:24.242558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7c42d-b4ed-43f7-b571-7ecf6af3b341 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:24.250144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae05d4-48b6-4a5b-a651-99faa7ff39da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:24.250817 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae05d4-48b6-4a5b-a651-99faa7ff39da admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:24.251095 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae05d4-48b6-4a5b-a651-99faa7ff39da admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:24.251412 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae05d4-48b6-4a5b-a651-99faa7ff39da admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:24.252061 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faae05d4-48b6-4a5b-a651-99faa7ff39da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:24.254655 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faae05d4-48b6-4a5b-a651-99faa7ff39da admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:24.261306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 776/1567] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:24 2022] POST /identity/v3/projects => generated 385 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:24.262628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c87d0-6488-46c8-97a6-9b4be8eb8010 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:24.269472 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 792/1568] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:24.273367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae05d4-48b6-4a5b-a651-99faa7ff39da admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:24.287271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae05d4-48b6-4a5b-a651-99faa7ff39da admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:24.415034 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c87d0-6488-46c8-97a6-9b4be8eb8010 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:24.431327 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a215b40-a963-4472-ad08-fdacb678f5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:24.432325 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a215b40-a963-4472-ad08-fdacb678f5e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:24.434308 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a215b40-a963-4472-ad08-fdacb678f5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:24.435114 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a215b40-a963-4472-ad08-fdacb678f5e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:24.435998 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a215b40-a963-4472-ad08-fdacb678f5e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:24.438364 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a215b40-a963-4472-ad08-fdacb678f5e0 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:24.463915 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c46b1-2530-4c78-9570-277d2e686dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:24.465074 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c46b1-2530-4c78-9570-277d2e686dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:24.465350 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c46b1-2530-4c78-9570-277d2e686dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:24.465647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c46b1-2530-4c78-9570-277d2e686dc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:24.466295 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c46b1-2530-4c78-9570-277d2e686dc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:24.469798 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c46b1-2530-4c78-9570-277d2e686dc8 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:24.482209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 777/1569] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:24 2022] POST /identity/v3/users => generated 299 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:24.483780 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a215b40-a963-4472-ad08-fdacb678f5e0 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:24.494908 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 793/1570] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:24.499064 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c46b1-2530-4c78-9570-277d2e686dc8 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:24.500732 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a215b40-a963-4472-ad08-fdacb678f5e0 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:24.513804 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978a8c8-007a-4938-a09b-9b06b0009608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:24.515238 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978a8c8-007a-4938-a09b-9b06b0009608 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:24.516116 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978a8c8-007a-4938-a09b-9b06b0009608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:24.516645 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978a8c8-007a-4938-a09b-9b06b0009608 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:24.518223 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b978a8c8-007a-4938-a09b-9b06b0009608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:24.520406 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c46b1-2530-4c78-9570-277d2e686dc8 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:24.521255 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b978a8c8-007a-4938-a09b-9b06b0009608 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:24.535423 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1fa09-3e9f-496a-84af-2355309c7d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:24.536186 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1fa09-3e9f-496a-84af-2355309c7d26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:24.536467 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1fa09-3e9f-496a-84af-2355309c7d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:24.536783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1fa09-3e9f-496a-84af-2355309c7d26 admin admin] PATH_INFO: `/v3/projects/6f19cee66f3b45e4b6d43e201dc79817/users/6d589205710044bf84a3bba940ac57ec/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:24.543889 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 778/1571] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:05:24.548072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978a8c8-007a-4938-a09b-9b06b0009608 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:24.572144 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978a8c8-007a-4938-a09b-9b06b0009608 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:24.579196 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39238e-55f2-4a68-a8c1-254fef3178df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:24.580258 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39238e-55f2-4a68-a8c1-254fef3178df admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:24.581142 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39238e-55f2-4a68-a8c1-254fef3178df admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:24.581948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39238e-55f2-4a68-a8c1-254fef3178df admin admin] PATH_INFO: `/v3/projects/7e1010bbafd34c7296d6b3416439815c/users/5508663f152442ca84f0e7f4e38efc72/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:24.582102 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1fa09-3e9f-496a-84af-2355309c7d26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f19cee66f3b45e4b6d43e201dc79817, user_id=6d589205710044bf84a3bba940ac57ec, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:24.584635 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1fa09-3e9f-496a-84af-2355309c7d26 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:24.625232 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 794/1572] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:24 2022] PUT /identity/v3/projects/6f19cee66f3b45e4b6d43e201dc79817/users/6d589205710044bf84a3bba940ac57ec/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:24.628964 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20f4a5-5d57-4630-8f52-73b9716871f7 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:24.628964 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20f4a5-5d57-4630-8f52-73b9716871f7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:24.629193 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20f4a5-5d57-4630-8f52-73b9716871f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:24.658887 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39238e-55f2-4a68-a8c1-254fef3178df admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e1010bbafd34c7296d6b3416439815c, user_id=5508663f152442ca84f0e7f4e38efc72, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:24.667859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39238e-55f2-4a68-a8c1-254fef3178df admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:24.680299 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-bb20f4a5-5d57-4630-8f52-73b9716871f7 None None] MFA Rules not processed for user `6d589205710044bf84a3bba940ac57ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:24.681815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-bb20f4a5-5d57-4630-8f52-73b9716871f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:24.715472 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 779/1573] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:24 2022] PUT /identity/v3/projects/7e1010bbafd34c7296d6b3416439815c/users/5508663f152442ca84f0e7f4e38efc72/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:05:24.720608 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39238e-55f2-4a68-a8c1-254fef3178df admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:24.754312 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:24.796736 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 795/1574] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:05:24.799467 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20f4a5-5d57-4630-8f52-73b9716871f7 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:24.908838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39238e-55f2-4a68-a8c1-254fef3178df admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:24.915989 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629a3e5-c004-487f-aee8-3f916e634d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:24.916867 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629a3e5-c004-487f-aee8-3f916e634d42 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:24.917239 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629a3e5-c004-487f-aee8-3f916e634d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:24.917756 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629a3e5-c004-487f-aee8-3f916e634d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:24.918493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629a3e5-c004-487f-aee8-3f916e634d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:24.920565 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629a3e5-c004-487f-aee8-3f916e634d42 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:24.957118 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 780/1575] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:24.961476 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629a3e5-c004-487f-aee8-3f916e634d42 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:24.963684 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20f4a5-5d57-4630-8f52-73b9716871f7 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:24.978733 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc48de7-e254-48da-bc01-217bf6ad43e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:24.979418 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc48de7-e254-48da-bc01-217bf6ad43e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:24.979704 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc48de7-e254-48da-bc01-217bf6ad43e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:24.980117 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc48de7-e254-48da-bc01-217bf6ad43e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:24.981956 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc48de7-e254-48da-bc01-217bf6ad43e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:24.986361 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc48de7-e254-48da-bc01-217bf6ad43e0 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:24.989784 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629a3e5-c004-487f-aee8-3f916e634d42 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:25.005569 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecebe15-8423-489f-9b90-2be04b26ca99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:25.006381 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecebe15-8423-489f-9b90-2be04b26ca99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:25.006789 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecebe15-8423-489f-9b90-2be04b26ca99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:25.007208 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecebe15-8423-489f-9b90-2be04b26ca99 admin admin] PATH_INFO: `/v3/projects/7e1010bbafd34c7296d6b3416439815c/users/5508663f152442ca84f0e7f4e38efc72/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:25.012775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecebe15-8423-489f-9b90-2be04b26ca99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e1010bbafd34c7296d6b3416439815c, user_id=5508663f152442ca84f0e7f4e38efc72, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:25.019713 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecebe15-8423-489f-9b90-2be04b26ca99 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:25.028047 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:25.035252 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 781/1576] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:24 2022] PUT /identity/v3/projects/7e1010bbafd34c7296d6b3416439815c/users/5508663f152442ca84f0e7f4e38efc72/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:25.046614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6b89f-3169-48eb-b6d1-5745e57716fb None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:25.047127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6b89f-3169-48eb-b6d1-5745e57716fb None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:25.047849 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6b89f-3169-48eb-b6d1-5745e57716fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:25.058770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 796/1577] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:24 2022] POST /identity/v3/projects => generated 365 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:25.062307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc48de7-e254-48da-bc01-217bf6ad43e0 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:25.103142 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-44e6b89f-3169-48eb-b6d1-5745e57716fb None None] MFA Rules not processed for user `5508663f152442ca84f0e7f4e38efc72`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:25.104846 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-44e6b89f-3169-48eb-b6d1-5745e57716fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:25.197864 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc48de7-e254-48da-bc01-217bf6ad43e0 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:25.206406 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899ae7c-3321-4f81-9f84-1d9211795da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:25.207259 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899ae7c-3321-4f81-9f84-1d9211795da5 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:25.207492 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899ae7c-3321-4f81-9f84-1d9211795da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:25.207730 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899ae7c-3321-4f81-9f84-1d9211795da5 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:25.208942 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899ae7c-3321-4f81-9f84-1d9211795da5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:25.211160 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899ae7c-3321-4f81-9f84-1d9211795da5 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:25.243470 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 782/1578] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:25.264940 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 797/1579] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:25 2022] POST /identity/v3/users => generated 289 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:25.270293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e6b89f-3169-48eb-b6d1-5745e57716fb None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:25.276674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899ae7c-3321-4f81-9f84-1d9211795da5 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:25.281842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6899ae7c-3321-4f81-9f84-1d9211795da5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:25.294890 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e6b89f-3169-48eb-b6d1-5745e57716fb None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:25.311545 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c90e87-52a1-473c-a1fe-11587abc8678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:25.314444 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c90e87-52a1-473c-a1fe-11587abc8678 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:25.316635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c90e87-52a1-473c-a1fe-11587abc8678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:25.318288 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c90e87-52a1-473c-a1fe-11587abc8678 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:25.319346 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c90e87-52a1-473c-a1fe-11587abc8678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:25.325401 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c90e87-52a1-473c-a1fe-11587abc8678 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:25.352753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 783/1580] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:25.357966 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c90e87-52a1-473c-a1fe-11587abc8678 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:25.372167 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c90e87-52a1-473c-a1fe-11587abc8678 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:25.378592 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a7637-db31-4b6d-b8a8-22dcb39dc10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:25.379284 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a7637-db31-4b6d-b8a8-22dcb39dc10e admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:25.380898 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a7637-db31-4b6d-b8a8-22dcb39dc10e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:25.380898 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a7637-db31-4b6d-b8a8-22dcb39dc10e admin admin] PATH_INFO: `/v3/projects/c9e6e5127d5242d2b4fb02f88eeb4d15/users/369e9d33640348508fc33b9dad1ab576/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:25.411639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a7637-db31-4b6d-b8a8-22dcb39dc10e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9e6e5127d5242d2b4fb02f88eeb4d15, user_id=369e9d33640348508fc33b9dad1ab576, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:25.413695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899ae7c-3321-4f81-9f84-1d9211795da5 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:25.414275 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a7637-db31-4b6d-b8a8-22dcb39dc10e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:25.427090 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b487ab-fa6d-4190-9f7a-2456fdf42d0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:25.433064 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b487ab-fa6d-4190-9f7a-2456fdf42d0e service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:25.433382 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b487ab-fa6d-4190-9f7a-2456fdf42d0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:25.434069 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b487ab-fa6d-4190-9f7a-2456fdf42d0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:25.437485 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d7b487ab-fa6d-4190-9f7a-2456fdf42d0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:25.452457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 784/1581] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:25 2022] PUT /identity/v3/projects/c9e6e5127d5242d2b4fb02f88eeb4d15/users/369e9d33640348508fc33b9dad1ab576/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:25.454408 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a7637-db31-4b6d-b8a8-22dcb39dc10e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:25.491030 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:25.584425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b487ab-fa6d-4190-9f7a-2456fdf42d0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:25.586174 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b487ab-fa6d-4190-9f7a-2456fdf42d0e service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:25.596781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 798/1582] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:25.605371 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a7637-db31-4b6d-b8a8-22dcb39dc10e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:25.613064 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b07f6-6112-46e5-9b22-98af626637a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:25.613666 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b07f6-6112-46e5-9b22-98af626637a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:25.613946 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b07f6-6112-46e5-9b22-98af626637a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:25.614206 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b07f6-6112-46e5-9b22-98af626637a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:25.614857 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12b07f6-6112-46e5-9b22-98af626637a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:25.617743 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12b07f6-6112-46e5-9b22-98af626637a9 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:25.632613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 785/1583] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:25.638976 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b487ab-fa6d-4190-9f7a-2456fdf42d0e service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:25.650143 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b487ab-fa6d-4190-9f7a-2456fdf42d0e service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:25.656085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472bf378-82f3-40af-a2bf-6f0819df5440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:25.656787 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472bf378-82f3-40af-a2bf-6f0819df5440 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:25.657118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472bf378-82f3-40af-a2bf-6f0819df5440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:25.657615 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472bf378-82f3-40af-a2bf-6f0819df5440 admin admin] PATH_INFO: `/v3/projects/c9e6e5127d5242d2b4fb02f88eeb4d15/users/369e9d33640348508fc33b9dad1ab576/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:25.659334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472bf378-82f3-40af-a2bf-6f0819df5440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9e6e5127d5242d2b4fb02f88eeb4d15, user_id=369e9d33640348508fc33b9dad1ab576, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:25.677026 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472bf378-82f3-40af-a2bf-6f0819df5440 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:25.700443 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 799/1584] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:25 2022] PUT /identity/v3/projects/c9e6e5127d5242d2b4fb02f88eeb4d15/users/369e9d33640348508fc33b9dad1ab576/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:25.702667 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2b3f4-69ea-4947-a582-fa066b38015f None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:25.702953 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2b3f4-69ea-4947-a582-fa066b38015f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:25.703234 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2b3f4-69ea-4947-a582-fa066b38015f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:25.757230 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-a3c2b3f4-69ea-4947-a582-fa066b38015f None None] MFA Rules not processed for user `369e9d33640348508fc33b9dad1ab576`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:25.772722 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a3c2b3f4-69ea-4947-a582-fa066b38015f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:25.966957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:26.030699 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 786/1585] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:26.102255 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472bf378-82f3-40af-a2bf-6f0819df5440 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:26.177871 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472bf378-82f3-40af-a2bf-6f0819df5440 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:26.182968 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f0c07-5b2e-4324-9495-3b0bc73566e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:26.182968 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f0c07-5b2e-4324-9495-3b0bc73566e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:26.182968 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f0c07-5b2e-4324-9495-3b0bc73566e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:26.183267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f0c07-5b2e-4324-9495-3b0bc73566e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:26.197120 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f0c07-5b2e-4324-9495-3b0bc73566e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:26.198744 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f0c07-5b2e-4324-9495-3b0bc73566e1 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:26.211097 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 800/1586] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:26.327700 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2b3f4-69ea-4947-a582-fa066b38015f None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:26.351825 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2b3f4-69ea-4947-a582-fa066b38015f None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:26.356151 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0c6e5-0b1a-41a7-a2cd-e514a763f97a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:26.356867 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0c6e5-0b1a-41a7-a2cd-e514a763f97a service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:26.357140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0c6e5-0b1a-41a7-a2cd-e514a763f97a service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:26.357427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0c6e5-0b1a-41a7-a2cd-e514a763f97a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:26.430980 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0c6e5-0b1a-41a7-a2cd-e514a763f97a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:26.432406 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0c6e5-0b1a-41a7-a2cd-e514a763f97a service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:26.442024 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 787/1587] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:27.318233 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f0c07-5b2e-4324-9495-3b0bc73566e1 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:27.319644 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-920f0c07-5b2e-4324-9495-3b0bc73566e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:27.330371 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f0c07-5b2e-4324-9495-3b0bc73566e1 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:27.334480 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce299bd-6e03-42b9-a31a-5f93dc87e153 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:27.335064 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce299bd-6e03-42b9-a31a-5f93dc87e153 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:27.335335 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce299bd-6e03-42b9-a31a-5f93dc87e153 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:27.335601 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce299bd-6e03-42b9-a31a-5f93dc87e153 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:27.391742 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce299bd-6e03-42b9-a31a-5f93dc87e153 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:27.393228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce299bd-6e03-42b9-a31a-5f93dc87e153 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:27.403538 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 801/1588] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:27.463689 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0c6e5-0b1a-41a7-a2cd-e514a763f97a service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:27.465213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7df0c6e5-0b1a-41a7-a2cd-e514a763f97a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:27.538313 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0c6e5-0b1a-41a7-a2cd-e514a763f97a service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:27.553534 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b8850-a6f5-4fad-b57b-f562033ee049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:27.557269 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b8850-a6f5-4fad-b57b-f562033ee049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:27.557550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b8850-a6f5-4fad-b57b-f562033ee049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:27.557883 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b8850-a6f5-4fad-b57b-f562033ee049 admin admin] PATH_INFO: `/v3/users/b4bf01bb6046421ea05f3e40a8b36900` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:27.559206 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b8850-a6f5-4fad-b57b-f562033ee049 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4bf01bb6046421ea05f3e40a8b36900)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:27.572678 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b8850-a6f5-4fad-b57b-f562033ee049 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:27.666311 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-963b8850-a6f5-4fad-b57b-f562033ee049 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4bf01bb6046421ea05f3e40a8b36900'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:27.667009 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-963b8850-a6f5-4fad-b57b-f562033ee049 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4bf01bb6046421ea05f3e40a8b36900'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:27.674788 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-963b8850-a6f5-4fad-b57b-f562033ee049 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4bf01bb6046421ea05f3e40a8b36900'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:27.690894 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:27.692579 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-963b8850-a6f5-4fad-b57b-f562033ee049 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4bf01bb6046421ea05f3e40a8b36900'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:27.703036 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-963b8850-a6f5-4fad-b57b-f562033ee049 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4bf01bb6046421ea05f3e40a8b36900'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:27.713030 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 788/1589] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:27 2022] DELETE /identity/v3/users/b4bf01bb6046421ea05f3e40a8b36900 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:27.718475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e37b4-29ad-4822-86e5-79537227aa7f None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:27.718851 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e37b4-29ad-4822-86e5-79537227aa7f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:27.719224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e37b4-29ad-4822-86e5-79537227aa7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:27.748088 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-7c4e37b4-29ad-4822-86e5-79537227aa7f None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:27.749793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7c4e37b4-29ad-4822-86e5-79537227aa7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:27.813537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:27.843834 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 802/1590] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:27.853327 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b8850-a6f5-4fad-b57b-f562033ee049 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:27.854800 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-963b8850-a6f5-4fad-b57b-f562033ee049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:27.966144 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b8850-a6f5-4fad-b57b-f562033ee049 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:27.976316 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e753c5-43ee-411e-a2c7-551d0da66675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:27.980589 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e753c5-43ee-411e-a2c7-551d0da66675 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:27.980972 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e753c5-43ee-411e-a2c7-551d0da66675 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:27.981199 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e753c5-43ee-411e-a2c7-551d0da66675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:28.000390 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e753c5-43ee-411e-a2c7-551d0da66675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:28.001941 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e753c5-43ee-411e-a2c7-551d0da66675 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:28.033245 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 789/1591] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:28.200128 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e37b4-29ad-4822-86e5-79537227aa7f None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:28.201987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7c4e37b4-29ad-4822-86e5-79537227aa7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:28.283601 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e37b4-29ad-4822-86e5-79537227aa7f None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:28.287446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a55bad-ab68-45bf-9112-86808066fafa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:28.288063 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a55bad-ab68-45bf-9112-86808066fafa service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:28.288337 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a55bad-ab68-45bf-9112-86808066fafa service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:28.288669 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a55bad-ab68-45bf-9112-86808066fafa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:28.291292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-31a55bad-ab68-45bf-9112-86808066fafa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:28.432175 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a55bad-ab68-45bf-9112-86808066fafa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:28.434991 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a55bad-ab68-45bf-9112-86808066fafa service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:28.462020 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 803/1592] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:28 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:28.691635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e753c5-43ee-411e-a2c7-551d0da66675 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:28.693766 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-53e753c5-43ee-411e-a2c7-551d0da66675 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:28.719414 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e753c5-43ee-411e-a2c7-551d0da66675 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:28.725230 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ee20a-abab-43b2-9095-62f5c2fbd6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:28.726747 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ee20a-abab-43b2-9095-62f5c2fbd6ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:28.727336 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ee20a-abab-43b2-9095-62f5c2fbd6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:28.727819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ee20a-abab-43b2-9095-62f5c2fbd6ef admin admin] PATH_INFO: `/v3/projects/8378e1b787a849da80075035ae75f25a` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:28.729349 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779ee20a-abab-43b2-9095-62f5c2fbd6ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=8378e1b787a849da80075035ae75f25a)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:28.732839 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779ee20a-abab-43b2-9095-62f5c2fbd6ef admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:28.788451 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-779ee20a-abab-43b2-9095-62f5c2fbd6ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8378e1b787a849da80075035ae75f25a', 'request_id': 'req-779ee20a-abab-43b2-9095-62f5c2fbd6ef'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:28.794275 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-779ee20a-abab-43b2-9095-62f5c2fbd6ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8378e1b787a849da80075035ae75f25a', 'request_id': 'req-779ee20a-abab-43b2-9095-62f5c2fbd6ef'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:28.801834 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-779ee20a-abab-43b2-9095-62f5c2fbd6ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8378e1b787a849da80075035ae75f25a', 'request_id': 'req-779ee20a-abab-43b2-9095-62f5c2fbd6ef'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:28.815274 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-779ee20a-abab-43b2-9095-62f5c2fbd6ef admin admin] The token cache is being invalidate because project 8378e1b787a849da80075035ae75f25a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:28.815274 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-779ee20a-abab-43b2-9095-62f5c2fbd6ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:28.816541 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-779ee20a-abab-43b2-9095-62f5c2fbd6ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:28.818106 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 790/1593] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:28 2022] DELETE /identity/v3/projects/8378e1b787a849da80075035ae75f25a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:28.822303 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a55bad-ab68-45bf-9112-86808066fafa service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:28.834847 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a55bad-ab68-45bf-9112-86808066fafa service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:28.915030 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1ef0a-4aa1-4d1c-8487-771ed53ecdf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:28.915754 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1ef0a-4aa1-4d1c-8487-771ed53ecdf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:28.915950 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1ef0a-4aa1-4d1c-8487-771ed53ecdf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:28.916237 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1ef0a-4aa1-4d1c-8487-771ed53ecdf1 admin admin] PATH_INFO: `/v3/users/b320b593ffb7431086fdc069ff67a33e` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:28.917532 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1ef0a-4aa1-4d1c-8487-771ed53ecdf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b320b593ffb7431086fdc069ff67a33e)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:28.919563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1ef0a-4aa1-4d1c-8487-771ed53ecdf1 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:28.969036 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:28.975714 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-14e1ef0a-4aa1-4d1c-8487-771ed53ecdf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b320b593ffb7431086fdc069ff67a33e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:28.976645 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-14e1ef0a-4aa1-4d1c-8487-771ed53ecdf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b320b593ffb7431086fdc069ff67a33e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:28.985838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-14e1ef0a-4aa1-4d1c-8487-771ed53ecdf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b320b593ffb7431086fdc069ff67a33e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:28.999356 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-14e1ef0a-4aa1-4d1c-8487-771ed53ecdf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b320b593ffb7431086fdc069ff67a33e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:29.000055 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-14e1ef0a-4aa1-4d1c-8487-771ed53ecdf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b320b593ffb7431086fdc069ff67a33e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:29.013059 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 804/1594] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:28 2022] DELETE /identity/v3/users/b320b593ffb7431086fdc069ff67a33e => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:29.401311 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ee20a-abab-43b2-9095-62f5c2fbd6ef admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:29.403210 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-779ee20a-abab-43b2-9095-62f5c2fbd6ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:29.419540 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ee20a-abab-43b2-9095-62f5c2fbd6ef admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:29.495785 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442512bf-64f5-4963-8376-e98b1e485f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:29.496549 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442512bf-64f5-4963-8376-e98b1e485f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:29.496862 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442512bf-64f5-4963-8376-e98b1e485f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:29.497432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442512bf-64f5-4963-8376-e98b1e485f60 admin admin] PATH_INFO: `/v3/projects/d8f9b113d0bb44c6a5e4f30e6b529600` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:29.498721 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442512bf-64f5-4963-8376-e98b1e485f60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8f9b113d0bb44c6a5e4f30e6b529600)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:29.500857 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442512bf-64f5-4963-8376-e98b1e485f60 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:29.522815 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:29.564188 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-442512bf-64f5-4963-8376-e98b1e485f60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8f9b113d0bb44c6a5e4f30e6b529600', 'request_id': 'req-442512bf-64f5-4963-8376-e98b1e485f60'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:29.570999 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-442512bf-64f5-4963-8376-e98b1e485f60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8f9b113d0bb44c6a5e4f30e6b529600', 'request_id': 'req-442512bf-64f5-4963-8376-e98b1e485f60'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:29.579821 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-442512bf-64f5-4963-8376-e98b1e485f60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8f9b113d0bb44c6a5e4f30e6b529600', 'request_id': 'req-442512bf-64f5-4963-8376-e98b1e485f60'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:29.595945 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-442512bf-64f5-4963-8376-e98b1e485f60 admin admin] The token cache is being invalidate because project d8f9b113d0bb44c6a5e4f30e6b529600 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:29.597118 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-442512bf-64f5-4963-8376-e98b1e485f60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:29.597819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-442512bf-64f5-4963-8376-e98b1e485f60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:29.602579 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 791/1595] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:29 2022] DELETE /identity/v3/projects/d8f9b113d0bb44c6a5e4f30e6b529600 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:29.603819 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1ef0a-4aa1-4d1c-8487-771ed53ecdf1 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:29.606140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-14e1ef0a-4aa1-4d1c-8487-771ed53ecdf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:29.609878 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442512bf-64f5-4963-8376-e98b1e485f60 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:29.613594 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-442512bf-64f5-4963-8376-e98b1e485f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:29.630174 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1ef0a-4aa1-4d1c-8487-771ed53ecdf1 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:29.732367 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442512bf-64f5-4963-8376-e98b1e485f60 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:29.767319 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83212702-3269-4425-98fb-a34fde9db31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:29.769301 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf1b15f-9130-4609-82bd-33c2ea903188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:29.769996 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1b15f-9130-4609-82bd-33c2ea903188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:29.770199 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83212702-3269-4425-98fb-a34fde9db31e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:29.770310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1b15f-9130-4609-82bd-33c2ea903188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:29.770857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1b15f-9130-4609-82bd-33c2ea903188 admin admin] PATH_INFO: `/v3/users/6d382677274e431e82b31a49bfddc382` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:29.771286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83212702-3269-4425-98fb-a34fde9db31e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:29.772608 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf1b15f-9130-4609-82bd-33c2ea903188 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d382677274e431e82b31a49bfddc382)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:29.773053 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83212702-3269-4425-98fb-a34fde9db31e admin admin] PATH_INFO: `/v3/users/6d589205710044bf84a3bba940ac57ec` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:29.774600 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf1b15f-9130-4609-82bd-33c2ea903188 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:29.776520 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83212702-3269-4425-98fb-a34fde9db31e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d589205710044bf84a3bba940ac57ec)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:29.783696 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83212702-3269-4425-98fb-a34fde9db31e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:29.852177 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-83212702-3269-4425-98fb-a34fde9db31e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d589205710044bf84a3bba940ac57ec'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:29.853113 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-83212702-3269-4425-98fb-a34fde9db31e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d589205710044bf84a3bba940ac57ec'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:29.867804 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-83212702-3269-4425-98fb-a34fde9db31e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d589205710044bf84a3bba940ac57ec'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:29.892410 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5cf1b15f-9130-4609-82bd-33c2ea903188 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d382677274e431e82b31a49bfddc382'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:29.893369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5cf1b15f-9130-4609-82bd-33c2ea903188 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d382677274e431e82b31a49bfddc382'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:29.896724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-83212702-3269-4425-98fb-a34fde9db31e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d589205710044bf84a3bba940ac57ec'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:29.904805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5cf1b15f-9130-4609-82bd-33c2ea903188 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d382677274e431e82b31a49bfddc382'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:29.905938 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-83212702-3269-4425-98fb-a34fde9db31e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d589205710044bf84a3bba940ac57ec'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:29.916936 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 792/1596] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:29 2022] DELETE /identity/v3/users/6d589205710044bf84a3bba940ac57ec => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:29.921953 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02293027-f2ff-452f-92b3-da7278ebdd84 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:29.922341 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02293027-f2ff-452f-92b3-da7278ebdd84 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:29.923179 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02293027-f2ff-452f-92b3-da7278ebdd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:29.939637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5cf1b15f-9130-4609-82bd-33c2ea903188 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d382677274e431e82b31a49bfddc382'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:29.940445 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5cf1b15f-9130-4609-82bd-33c2ea903188 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d382677274e431e82b31a49bfddc382'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:29.950028 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-02293027-f2ff-452f-92b3-da7278ebdd84 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:29.951498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-02293027-f2ff-452f-92b3-da7278ebdd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:29.960318 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 805/1597] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:29 2022] DELETE /identity/v3/users/6d382677274e431e82b31a49bfddc382 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:29.981694 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:30.062473 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 793/1598] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:30.077257 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf1b15f-9130-4609-82bd-33c2ea903188 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:30.079071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5cf1b15f-9130-4609-82bd-33c2ea903188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:30.138459 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:30.163179 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf1b15f-9130-4609-82bd-33c2ea903188 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:30.167870 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d1e6b-c4fe-4d63-b620-7f7fb44a611e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:30.168609 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d1e6b-c4fe-4d63-b620-7f7fb44a611e service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:30.169012 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d1e6b-c4fe-4d63-b620-7f7fb44a611e service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:30.169360 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d1e6b-c4fe-4d63-b620-7f7fb44a611e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:30.292027 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d1e6b-c4fe-4d63-b620-7f7fb44a611e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:30.293650 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d1e6b-c4fe-4d63-b620-7f7fb44a611e service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:30.311621 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 806/1599] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:30.823011 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02293027-f2ff-452f-92b3-da7278ebdd84 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:30.824445 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-02293027-f2ff-452f-92b3-da7278ebdd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:30.843071 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02293027-f2ff-452f-92b3-da7278ebdd84 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:30.851971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310fa77-1344-49e9-a54a-2fcde0c73bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:30.852635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310fa77-1344-49e9-a54a-2fcde0c73bab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:30.853043 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310fa77-1344-49e9-a54a-2fcde0c73bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:30.853449 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310fa77-1344-49e9-a54a-2fcde0c73bab admin admin] PATH_INFO: `/v3/projects/a493ad3ccaf341cda57ef1070ffd11f9` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:30.856097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7310fa77-1344-49e9-a54a-2fcde0c73bab admin admin] RBAC: Authorizing `identity:delete_project(project_id=a493ad3ccaf341cda57ef1070ffd11f9)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:30.858487 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7310fa77-1344-49e9-a54a-2fcde0c73bab admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:30.933823 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7310fa77-1344-49e9-a54a-2fcde0c73bab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a493ad3ccaf341cda57ef1070ffd11f9', 'request_id': 'req-7310fa77-1344-49e9-a54a-2fcde0c73bab'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:30.943243 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7310fa77-1344-49e9-a54a-2fcde0c73bab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a493ad3ccaf341cda57ef1070ffd11f9', 'request_id': 'req-7310fa77-1344-49e9-a54a-2fcde0c73bab'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:30.953208 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7310fa77-1344-49e9-a54a-2fcde0c73bab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a493ad3ccaf341cda57ef1070ffd11f9', 'request_id': 'req-7310fa77-1344-49e9-a54a-2fcde0c73bab'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:30.983900 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7310fa77-1344-49e9-a54a-2fcde0c73bab admin admin] The token cache is being invalidate because project a493ad3ccaf341cda57ef1070ffd11f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:30.984301 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7310fa77-1344-49e9-a54a-2fcde0c73bab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:30.985068 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7310fa77-1344-49e9-a54a-2fcde0c73bab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:30.988744 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 794/1600] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:30 2022] DELETE /identity/v3/projects/a493ad3ccaf341cda57ef1070ffd11f9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:30.992472 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a9231-cfe8-4308-bdc7-95ca724c9bcf None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:30.993323 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a9231-cfe8-4308-bdc7-95ca724c9bcf None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:30.993794 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a9231-cfe8-4308-bdc7-95ca724c9bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:30.999865 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310fa77-1344-49e9-a54a-2fcde0c73bab admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:31.006044 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7310fa77-1344-49e9-a54a-2fcde0c73bab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:31.038475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-260a9231-cfe8-4308-bdc7-95ca724c9bcf None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:31.041958 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-260a9231-cfe8-4308-bdc7-95ca724c9bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:31.164362 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 807/1601] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:05:31.164463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310fa77-1344-49e9-a54a-2fcde0c73bab admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:31.167120 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc322dc-0cfc-474c-ab00-9a209d479691 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:31.167416 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc322dc-0cfc-474c-ab00-9a209d479691 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:31.167692 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc322dc-0cfc-474c-ab00-9a209d479691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:31.168225 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c3f63-c126-47bd-a8b0-84bd1702cea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:31.169101 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c3f63-c126-47bd-a8b0-84bd1702cea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:31.169401 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c3f63-c126-47bd-a8b0-84bd1702cea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:31.169776 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c3f63-c126-47bd-a8b0-84bd1702cea6 admin admin] PATH_INFO: `/v3/projects/6f19cee66f3b45e4b6d43e201dc79817` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:31.180451 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c3f63-c126-47bd-a8b0-84bd1702cea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f19cee66f3b45e4b6d43e201dc79817)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:31.182578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c3f63-c126-47bd-a8b0-84bd1702cea6 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:31.197433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-7dc322dc-0cfc-474c-ab00-9a209d479691 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:31.198766 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7dc322dc-0cfc-474c-ab00-9a209d479691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:31.213236 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 808/1602] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:31.218701 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc322dc-0cfc-474c-ab00-9a209d479691 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:31.229947 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc322dc-0cfc-474c-ab00-9a209d479691 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:31.235681 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7b1b6-640c-4779-a83a-3efb0d3ffa5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:31.236465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7b1b6-640c-4779-a83a-3efb0d3ffa5e admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:31.236963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7b1b6-640c-4779-a83a-3efb0d3ffa5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:31.237413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7b1b6-640c-4779-a83a-3efb0d3ffa5e admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:31.240451 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7b1b6-640c-4779-a83a-3efb0d3ffa5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:05:31.240991 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7b1b6-640c-4779-a83a-3efb0d3ffa5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:31.245373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7b1b6-640c-4779-a83a-3efb0d3ffa5e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:31.273126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 809/1603] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:05:31.274974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7b1b6-640c-4779-a83a-3efb0d3ffa5e admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:31.289869 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7b1b6-640c-4779-a83a-3efb0d3ffa5e admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:31.290981 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:31.291986 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7a6c3f63-c126-47bd-a8b0-84bd1702cea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f19cee66f3b45e4b6d43e201dc79817', 'request_id': 'req-7a6c3f63-c126-47bd-a8b0-84bd1702cea6'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:31.310038 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7a6c3f63-c126-47bd-a8b0-84bd1702cea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f19cee66f3b45e4b6d43e201dc79817', 'request_id': 'req-7a6c3f63-c126-47bd-a8b0-84bd1702cea6'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:31.327369 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7a6c3f63-c126-47bd-a8b0-84bd1702cea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f19cee66f3b45e4b6d43e201dc79817', 'request_id': 'req-7a6c3f63-c126-47bd-a8b0-84bd1702cea6'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:31.356420 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7a6c3f63-c126-47bd-a8b0-84bd1702cea6 admin admin] The token cache is being invalidate because project 6f19cee66f3b45e4b6d43e201dc79817 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:31.356743 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7a6c3f63-c126-47bd-a8b0-84bd1702cea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:31.357512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7a6c3f63-c126-47bd-a8b0-84bd1702cea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:31.359279 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 795/1604] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:30 2022] DELETE /identity/v3/projects/6f19cee66f3b45e4b6d43e201dc79817 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:31.363970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c3f63-c126-47bd-a8b0-84bd1702cea6 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:31.365835 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7a6c3f63-c126-47bd-a8b0-84bd1702cea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:31.424035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b38ba6-270b-49e0-a135-ee322a48e808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:31.425066 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b38ba6-270b-49e0-a135-ee322a48e808 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:31.425646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b38ba6-270b-49e0-a135-ee322a48e808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:31.425767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b38ba6-270b-49e0-a135-ee322a48e808 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:31.426790 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b38ba6-270b-49e0-a135-ee322a48e808 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:31.429141 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b38ba6-270b-49e0-a135-ee322a48e808 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:31.450266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:31.479311 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 810/1605] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:31 2022] POST /identity/v3/projects => generated 385 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:31.481533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b38ba6-270b-49e0-a135-ee322a48e808 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:31.489713 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f5b38ba6-270b-49e0-a135-ee322a48e808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:31.504370 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c3f63-c126-47bd-a8b0-84bd1702cea6 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:31.680316 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b38ba6-270b-49e0-a135-ee322a48e808 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:31.683917 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445dddf-342e-45ea-a723-66725ff10684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:31.684914 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445dddf-342e-45ea-a723-66725ff10684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:31.685489 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445dddf-342e-45ea-a723-66725ff10684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:31.685967 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445dddf-342e-45ea-a723-66725ff10684 admin admin] PATH_INFO: `/v3/users/369e9d33640348508fc33b9dad1ab576` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:31.687409 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8cf643-66cb-4660-8483-3ef0157b685d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:31.687751 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445dddf-342e-45ea-a723-66725ff10684 admin admin] RBAC: Authorizing `identity:delete_user(user_id=369e9d33640348508fc33b9dad1ab576)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:31.688032 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cf643-66cb-4660-8483-3ef0157b685d admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:31.688306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cf643-66cb-4660-8483-3ef0157b685d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:31.688573 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cf643-66cb-4660-8483-3ef0157b685d admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:31.689232 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8cf643-66cb-4660-8483-3ef0157b685d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:31.691229 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445dddf-342e-45ea-a723-66725ff10684 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:31.691539 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8cf643-66cb-4660-8483-3ef0157b685d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:31.759182 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 811/1606] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:31 2022] POST /identity/v3/users => generated 293 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:31.762960 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8cf643-66cb-4660-8483-3ef0157b685d admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:31.777901 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8cf643-66cb-4660-8483-3ef0157b685d admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:31.797998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517c3b9-9688-469c-8283-241f633ec675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:31.798709 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517c3b9-9688-469c-8283-241f633ec675 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:31.799971 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517c3b9-9688-469c-8283-241f633ec675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:31.800229 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517c3b9-9688-469c-8283-241f633ec675 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:31.802000 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517c3b9-9688-469c-8283-241f633ec675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:31.802646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517c3b9-9688-469c-8283-241f633ec675 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:31.824443 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7445dddf-342e-45ea-a723-66725ff10684 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '369e9d33640348508fc33b9dad1ab576'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:31.825155 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7445dddf-342e-45ea-a723-66725ff10684 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '369e9d33640348508fc33b9dad1ab576'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:31.826465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 812/1607] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:05:31.830003 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517c3b9-9688-469c-8283-241f633ec675 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:31.848510 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7445dddf-342e-45ea-a723-66725ff10684 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '369e9d33640348508fc33b9dad1ab576'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:31.857740 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517c3b9-9688-469c-8283-241f633ec675 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:31.871450 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a4033-4545-4e44-a4c0-114e3c6a4e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:31.872072 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a4033-4545-4e44-a4c0-114e3c6a4e92 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:31.872349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a4033-4545-4e44-a4c0-114e3c6a4e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:31.872633 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a4033-4545-4e44-a4c0-114e3c6a4e92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:31.873295 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8a4033-4545-4e44-a4c0-114e3c6a4e92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:31.875830 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8a4033-4545-4e44-a4c0-114e3c6a4e92 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:31.876904 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7445dddf-342e-45ea-a723-66725ff10684 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '369e9d33640348508fc33b9dad1ab576'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:31.891015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7445dddf-342e-45ea-a723-66725ff10684 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '369e9d33640348508fc33b9dad1ab576'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:31.895391 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 813/1608] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:31.902084 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a4033-4545-4e44-a4c0-114e3c6a4e92 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:31.904003 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 796/1609] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:31 2022] DELETE /identity/v3/users/369e9d33640348508fc33b9dad1ab576 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:05:31.908562 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2b8a4033-4545-4e44-a4c0-114e3c6a4e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:31.930153 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a4033-4545-4e44-a4c0-114e3c6a4e92 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:32.032313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ac502-4729-41e7-8214-217ed305937a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:32.035768 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ac502-4729-41e7-8214-217ed305937a admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:32.035768 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ac502-4729-41e7-8214-217ed305937a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:32.035768 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ac502-4729-41e7-8214-217ed305937a admin admin] PATH_INFO: `/v3/projects/ae45d83432b34e2aa9c24c60260b8ea1/users/cc9d8d424c9048fdacac71bc279467fa/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:32.059401 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ac502-4729-41e7-8214-217ed305937a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae45d83432b34e2aa9c24c60260b8ea1, user_id=cc9d8d424c9048fdacac71bc279467fa, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:32.063888 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ac502-4729-41e7-8214-217ed305937a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:32.089410 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 814/1610] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:31 2022] PUT /identity/v3/projects/ae45d83432b34e2aa9c24c60260b8ea1/users/cc9d8d424c9048fdacac71bc279467fa/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:32.101001 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f8e17-771d-4bf8-9a51-c172720de350 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:32.101378 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f8e17-771d-4bf8-9a51-c172720de350 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:32.101735 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f8e17-771d-4bf8-9a51-c172720de350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:32.133686 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:32.191046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-432f8e17-771d-4bf8-9a51-c172720de350 None None] MFA Rules not processed for user `cc9d8d424c9048fdacac71bc279467fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:32.192981 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-432f8e17-771d-4bf8-9a51-c172720de350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:32.317140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 797/1611] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:32.328098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ac502-4729-41e7-8214-217ed305937a admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:32.330222 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7d2ac502-4729-41e7-8214-217ed305937a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:32.393440 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:32.405890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ac502-4729-41e7-8214-217ed305937a admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:32.410987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11909d5c-5330-4419-a94f-46733eb54eb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:32.411915 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11909d5c-5330-4419-a94f-46733eb54eb6 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:32.411915 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11909d5c-5330-4419-a94f-46733eb54eb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:32.412383 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11909d5c-5330-4419-a94f-46733eb54eb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:32.422269 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11909d5c-5330-4419-a94f-46733eb54eb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:32.424133 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11909d5c-5330-4419-a94f-46733eb54eb6 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:32.435540 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 815/1612] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:32.677976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f8e17-771d-4bf8-9a51-c172720de350 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:32.679442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-432f8e17-771d-4bf8-9a51-c172720de350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:32.801232 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f8e17-771d-4bf8-9a51-c172720de350 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:32.809647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad1162-5b2a-4dc9-a173-f06bbba19fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:32.810315 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad1162-5b2a-4dc9-a173-f06bbba19fae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:32.810596 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad1162-5b2a-4dc9-a173-f06bbba19fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:32.810853 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad1162-5b2a-4dc9-a173-f06bbba19fae admin admin] PATH_INFO: `/v3/projects/c9e6e5127d5242d2b4fb02f88eeb4d15` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:32.812187 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad1162-5b2a-4dc9-a173-f06bbba19fae admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9e6e5127d5242d2b4fb02f88eeb4d15)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:32.814898 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad1162-5b2a-4dc9-a173-f06bbba19fae admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:32.896267 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-efad1162-5b2a-4dc9-a173-f06bbba19fae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9e6e5127d5242d2b4fb02f88eeb4d15', 'request_id': 'req-efad1162-5b2a-4dc9-a173-f06bbba19fae'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:32.904131 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-efad1162-5b2a-4dc9-a173-f06bbba19fae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9e6e5127d5242d2b4fb02f88eeb4d15', 'request_id': 'req-efad1162-5b2a-4dc9-a173-f06bbba19fae'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:32.911084 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-efad1162-5b2a-4dc9-a173-f06bbba19fae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9e6e5127d5242d2b4fb02f88eeb4d15', 'request_id': 'req-efad1162-5b2a-4dc9-a173-f06bbba19fae'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:32.929192 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-efad1162-5b2a-4dc9-a173-f06bbba19fae admin admin] The token cache is being invalidate because project c9e6e5127d5242d2b4fb02f88eeb4d15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:32.929482 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-efad1162-5b2a-4dc9-a173-f06bbba19fae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:32.931154 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-efad1162-5b2a-4dc9-a173-f06bbba19fae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:32.941450 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 798/1613] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:32 2022] DELETE /identity/v3/projects/c9e6e5127d5242d2b4fb02f88eeb4d15 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:32.946234 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4aee7d-2bb1-4ee2-af69-cf07aedb58ec None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:32.946524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4aee7d-2bb1-4ee2-af69-cf07aedb58ec None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:32.946821 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4aee7d-2bb1-4ee2-af69-cf07aedb58ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:32.971851 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-1b4aee7d-2bb1-4ee2-af69-cf07aedb58ec None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:32.973208 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1b4aee7d-2bb1-4ee2-af69-cf07aedb58ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:33.046092 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 816/1614] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:05:33.054784 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934dfb5-15fd-49fc-8f23-016cad49127c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:33.055061 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934dfb5-15fd-49fc-8f23-016cad49127c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:33.055494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934dfb5-15fd-49fc-8f23-016cad49127c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:33.077662 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c934dfb5-15fd-49fc-8f23-016cad49127c None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:33.079054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c934dfb5-15fd-49fc-8f23-016cad49127c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:33.092344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 799/1615] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:33.093169 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4aee7d-2bb1-4ee2-af69-cf07aedb58ec None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:33.104567 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4aee7d-2bb1-4ee2-af69-cf07aedb58ec None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:33.116572 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65827cb2-3709-4cd1-b64c-e8332f6e63f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:33.117346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65827cb2-3709-4cd1-b64c-e8332f6e63f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:33.117626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65827cb2-3709-4cd1-b64c-e8332f6e63f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:33.117899 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65827cb2-3709-4cd1-b64c-e8332f6e63f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:33.118544 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65827cb2-3709-4cd1-b64c-e8332f6e63f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:05:33.118847 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65827cb2-3709-4cd1-b64c-e8332f6e63f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:33.120973 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65827cb2-3709-4cd1-b64c-e8332f6e63f1 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:33.128250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934dfb5-15fd-49fc-8f23-016cad49127c None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:33.130004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c934dfb5-15fd-49fc-8f23-016cad49127c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:33.137312 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 817/1616] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 06:05:33.141094 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65827cb2-3709-4cd1-b64c-e8332f6e63f1 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:33.154093 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65827cb2-3709-4cd1-b64c-e8332f6e63f1 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:33.157213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934dfb5-15fd-49fc-8f23-016cad49127c None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:33.160751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df670f2-341d-4709-ac8c-23d8da1db509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:33.161459 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df670f2-341d-4709-ac8c-23d8da1db509 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:33.161644 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df670f2-341d-4709-ac8c-23d8da1db509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:33.161918 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df670f2-341d-4709-ac8c-23d8da1db509 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:33.162562 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df670f2-341d-4709-ac8c-23d8da1db509 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:33.164631 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df670f2-341d-4709-ac8c-23d8da1db509 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:33.173110 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98ae89-117c-4b8b-9774-d2ab164c4473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:33.173913 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98ae89-117c-4b8b-9774-d2ab164c4473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:33.174293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98ae89-117c-4b8b-9774-d2ab164c4473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:33.175368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98ae89-117c-4b8b-9774-d2ab164c4473 admin admin] PATH_INFO: `/v3/users/3d61d267f1024a77a294d3031db7e7c4` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:33.176759 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f98ae89-117c-4b8b-9774-d2ab164c4473 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d61d267f1024a77a294d3031db7e7c4)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:33.179432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f98ae89-117c-4b8b-9774-d2ab164c4473 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:33.194373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:33.206771 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 818/1617] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:33 2022] POST /identity/v3/projects => generated 367 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:05:33.211514 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df670f2-341d-4709-ac8c-23d8da1db509 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:33.259355 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8f98ae89-117c-4b8b-9774-d2ab164c4473 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d61d267f1024a77a294d3031db7e7c4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:33.260483 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8f98ae89-117c-4b8b-9774-d2ab164c4473 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d61d267f1024a77a294d3031db7e7c4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:33.277284 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8f98ae89-117c-4b8b-9774-d2ab164c4473 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d61d267f1024a77a294d3031db7e7c4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:33.298266 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8f98ae89-117c-4b8b-9774-d2ab164c4473 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d61d267f1024a77a294d3031db7e7c4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:33.311576 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8f98ae89-117c-4b8b-9774-d2ab164c4473 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d61d267f1024a77a294d3031db7e7c4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:33.322186 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 800/1618] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:33 2022] DELETE /identity/v3/users/3d61d267f1024a77a294d3031db7e7c4 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:05:33.328299 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292109f-6ab9-4471-9ee4-b53f55116831 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:33.328299 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292109f-6ab9-4471-9ee4-b53f55116831 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:33.328566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292109f-6ab9-4471-9ee4-b53f55116831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:33.365413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df670f2-341d-4709-ac8c-23d8da1db509 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:33.369237 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c292109f-6ab9-4471-9ee4-b53f55116831 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:33.371487 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c292109f-6ab9-4471-9ee4-b53f55116831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:33.444565 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dec67-e0bc-4a82-9da5-a27815267e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:33.445159 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dec67-e0bc-4a82-9da5-a27815267e0e admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:33.445444 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dec67-e0bc-4a82-9da5-a27815267e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:33.445718 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dec67-e0bc-4a82-9da5-a27815267e0e admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:33.446347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96dec67-e0bc-4a82-9da5-a27815267e0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:33.448276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96dec67-e0bc-4a82-9da5-a27815267e0e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:33.458145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:33.465458 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 801/1619] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:33.471675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 819/1620] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:33 2022] POST /identity/v3/users => generated 284 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:05:33.475390 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292109f-6ab9-4471-9ee4-b53f55116831 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:33.476433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dec67-e0bc-4a82-9da5-a27815267e0e admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:33.476973 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c292109f-6ab9-4471-9ee4-b53f55116831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:33.478047 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f96dec67-e0bc-4a82-9da5-a27815267e0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:33.489099 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292109f-6ab9-4471-9ee4-b53f55116831 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:33.493714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002d672-5082-467c-ac7b-5b2a45696920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:33.494317 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002d672-5082-467c-ac7b-5b2a45696920 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:33.494597 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002d672-5082-467c-ac7b-5b2a45696920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:33.494876 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002d672-5082-467c-ac7b-5b2a45696920 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:33.495509 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c002d672-5082-467c-ac7b-5b2a45696920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:33.497146 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c002d672-5082-467c-ac7b-5b2a45696920 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:33.508071 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 802/1621] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:33.512185 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002d672-5082-467c-ac7b-5b2a45696920 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:33.522930 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002d672-5082-467c-ac7b-5b2a45696920 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:33.528813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3e894-5ecd-474d-95a4-1fa12b5c927d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:33.529431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3e894-5ecd-474d-95a4-1fa12b5c927d admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:33.529697 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3e894-5ecd-474d-95a4-1fa12b5c927d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:33.529970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3e894-5ecd-474d-95a4-1fa12b5c927d admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:33.530628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3e894-5ecd-474d-95a4-1fa12b5c927d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:33.532208 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3e894-5ecd-474d-95a4-1fa12b5c927d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:33.536341 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dec67-e0bc-4a82-9da5-a27815267e0e admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:33.540001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ba346-51b7-4920-b3de-9b7377ff62e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:33.540611 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ba346-51b7-4920-b3de-9b7377ff62e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:33.540924 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ba346-51b7-4920-b3de-9b7377ff62e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:33.541220 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ba346-51b7-4920-b3de-9b7377ff62e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:33.542486 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 803/1622] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:33.546450 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3e894-5ecd-474d-95a4-1fa12b5c927d admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:33.550414 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ba346-51b7-4920-b3de-9b7377ff62e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:33.551854 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ba346-51b7-4920-b3de-9b7377ff62e4 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:33.555834 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3e894-5ecd-474d-95a4-1fa12b5c927d admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:33.560409 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c82195-a08f-4b77-9ece-2957b18e04fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:33.561080 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c82195-a08f-4b77-9ece-2957b18e04fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:33.561368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c82195-a08f-4b77-9ece-2957b18e04fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:33.561467 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 820/1623] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:05:33.561648 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c82195-a08f-4b77-9ece-2957b18e04fb admin admin] PATH_INFO: `/v3/projects/249dc6c053374604b414f32f290f6b28/users/aa1775c61bb94268ab327641b731de65/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:33.577103 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c82195-a08f-4b77-9ece-2957b18e04fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=249dc6c053374604b414f32f290f6b28, user_id=aa1775c61bb94268ab327641b731de65, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:33.579845 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c82195-a08f-4b77-9ece-2957b18e04fb admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:33.593744 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 804/1624] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:33 2022] PUT /identity/v3/projects/249dc6c053374604b414f32f290f6b28/users/aa1775c61bb94268ab327641b731de65/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:05:33.606314 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05669bc-ea09-432f-bdd3-fa1745547745 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:33.606601 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05669bc-ea09-432f-bdd3-fa1745547745 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:33.606880 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05669bc-ea09-432f-bdd3-fa1745547745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:33.644583 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d05669bc-ea09-432f-bdd3-fa1745547745 None None] MFA Rules not processed for user `aa1775c61bb94268ab327641b731de65`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:33.645863 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d05669bc-ea09-432f-bdd3-fa1745547745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:33.727987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 821/1625] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:05:33.733170 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c82195-a08f-4b77-9ece-2957b18e04fb admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:33.785252 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:33.807509 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c82195-a08f-4b77-9ece-2957b18e04fb admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:33.812246 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3ac50-fe17-434a-a678-c5fb8937f37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:33.812950 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3ac50-fe17-434a-a678-c5fb8937f37e admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:33.813231 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3ac50-fe17-434a-a678-c5fb8937f37e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:33.813503 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3ac50-fe17-434a-a678-c5fb8937f37e admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:33.814147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3ac50-fe17-434a-a678-c5fb8937f37e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:33.816653 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3ac50-fe17-434a-a678-c5fb8937f37e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:33.849385 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 805/1626] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:33 2022] POST /identity/v3/projects => generated 369 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:05:33.853666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05669bc-ea09-432f-bdd3-fa1745547745 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:33.866582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3ac50-fe17-434a-a678-c5fb8937f37e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:33.868380 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-71d3ac50-fe17-434a-a678-c5fb8937f37e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:33.869569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05669bc-ea09-432f-bdd3-fa1745547745 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:33.951727 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bf407-284e-46f1-8f2b-fe3de2d6b3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:33.952559 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bf407-284e-46f1-8f2b-fe3de2d6b3cc admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:33.952873 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bf407-284e-46f1-8f2b-fe3de2d6b3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:33.953146 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bf407-284e-46f1-8f2b-fe3de2d6b3cc admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:33.953779 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1bf407-284e-46f1-8f2b-fe3de2d6b3cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:33.954198 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3ac50-fe17-434a-a678-c5fb8937f37e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:33.955816 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1bf407-284e-46f1-8f2b-fe3de2d6b3cc admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:33.975077 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884a1f7-1dc2-48c6-9e69-8fae62f173f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:33.976190 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884a1f7-1dc2-48c6-9e69-8fae62f173f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:33.976375 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884a1f7-1dc2-48c6-9e69-8fae62f173f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:33.976623 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884a1f7-1dc2-48c6-9e69-8fae62f173f5 admin admin] PATH_INFO: `/v3/projects/a572c375448a4037aadfb45f679ea2fb` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:33.978089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884a1f7-1dc2-48c6-9e69-8fae62f173f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a572c375448a4037aadfb45f679ea2fb)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:33.980121 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884a1f7-1dc2-48c6-9e69-8fae62f173f5 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:33.983968 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 822/1627] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:33 2022] POST /identity/v3/users => generated 291 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:05:33.984943 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bf407-284e-46f1-8f2b-fe3de2d6b3cc admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:34.000626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bf407-284e-46f1-8f2b-fe3de2d6b3cc admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:34.008014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922040b7-35f3-4778-a3a6-fb0eb8be525a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:34.008954 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922040b7-35f3-4778-a3a6-fb0eb8be525a admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:34.009245 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922040b7-35f3-4778-a3a6-fb0eb8be525a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:34.009634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922040b7-35f3-4778-a3a6-fb0eb8be525a admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:34.010318 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922040b7-35f3-4778-a3a6-fb0eb8be525a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:34.012004 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922040b7-35f3-4778-a3a6-fb0eb8be525a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:34.021048 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:34.028644 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 823/1628] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:34.033271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922040b7-35f3-4778-a3a6-fb0eb8be525a admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:34.043431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2884a1f7-1dc2-48c6-9e69-8fae62f173f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a572c375448a4037aadfb45f679ea2fb', 'request_id': 'req-2884a1f7-1dc2-48c6-9e69-8fae62f173f5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:34.048621 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922040b7-35f3-4778-a3a6-fb0eb8be525a admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:34.055801 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2884a1f7-1dc2-48c6-9e69-8fae62f173f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a572c375448a4037aadfb45f679ea2fb', 'request_id': 'req-2884a1f7-1dc2-48c6-9e69-8fae62f173f5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:34.066316 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2884a1f7-1dc2-48c6-9e69-8fae62f173f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a572c375448a4037aadfb45f679ea2fb', 'request_id': 'req-2884a1f7-1dc2-48c6-9e69-8fae62f173f5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:34.085899 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2884a1f7-1dc2-48c6-9e69-8fae62f173f5 admin admin] The token cache is being invalidate because project a572c375448a4037aadfb45f679ea2fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:34.086203 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2884a1f7-1dc2-48c6-9e69-8fae62f173f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:34.086888 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2884a1f7-1dc2-48c6-9e69-8fae62f173f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:34.088635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 806/1629] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:33 2022] DELETE /identity/v3/projects/a572c375448a4037aadfb45f679ea2fb => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:34.092230 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884a1f7-1dc2-48c6-9e69-8fae62f173f5 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:34.095208 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2884a1f7-1dc2-48c6-9e69-8fae62f173f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:34.132814 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d54efc0-8eb3-49c7-9daa-a759deb02756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:34.133983 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d54efc0-8eb3-49c7-9daa-a759deb02756 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:34.134266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d54efc0-8eb3-49c7-9daa-a759deb02756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:34.134537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d54efc0-8eb3-49c7-9daa-a759deb02756 admin admin] PATH_INFO: `/v3/projects/f2653b2277a64b6191902e0cc0872543/users/5ea8755120d54fd397d7ce0f371b0557/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:34.151557 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d54efc0-8eb3-49c7-9daa-a759deb02756 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2653b2277a64b6191902e0cc0872543, user_id=5ea8755120d54fd397d7ce0f371b0557, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:34.154958 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d54efc0-8eb3-49c7-9daa-a759deb02756 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:34.164619 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:34.170467 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884a1f7-1dc2-48c6-9e69-8fae62f173f5 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:34.173497 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 824/1630] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:34 2022] PUT /identity/v3/projects/f2653b2277a64b6191902e0cc0872543/users/5ea8755120d54fd397d7ce0f371b0557/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:05:34.177786 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d54efc0-8eb3-49c7-9daa-a759deb02756 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:34.179908 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6d54efc0-8eb3-49c7-9daa-a759deb02756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:34.243490 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597935e-aa71-462f-ac95-a8be1c628506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:34.244114 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597935e-aa71-462f-ac95-a8be1c628506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:34.244323 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597935e-aa71-462f-ac95-a8be1c628506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:34.244594 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597935e-aa71-462f-ac95-a8be1c628506 admin admin] PATH_INFO: `/v3/users/70c3d673547c4f45945a97eb4921bac2` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:34.245766 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a597935e-aa71-462f-ac95-a8be1c628506 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70c3d673547c4f45945a97eb4921bac2)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:34.247678 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a597935e-aa71-462f-ac95-a8be1c628506 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:34.254644 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d54efc0-8eb3-49c7-9daa-a759deb02756 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:34.259580 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64478bfe-1bbf-4338-8358-ec8b8fd7a388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:34.260219 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64478bfe-1bbf-4338-8358-ec8b8fd7a388 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:34.260497 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64478bfe-1bbf-4338-8358-ec8b8fd7a388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:34.260813 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64478bfe-1bbf-4338-8358-ec8b8fd7a388 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:34.261452 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64478bfe-1bbf-4338-8358-ec8b8fd7a388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:34.263041 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64478bfe-1bbf-4338-8358-ec8b8fd7a388 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:34.273627 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 825/1631] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:05:34.277972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64478bfe-1bbf-4338-8358-ec8b8fd7a388 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:34.289286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64478bfe-1bbf-4338-8358-ec8b8fd7a388 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:34.294138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36f835-a8ca-427d-8856-4b7a1bd01d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:34.294909 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36f835-a8ca-427d-8856-4b7a1bd01d1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:34.295100 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36f835-a8ca-427d-8856-4b7a1bd01d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:34.295378 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36f835-a8ca-427d-8856-4b7a1bd01d1d admin admin] PATH_INFO: `/v3/projects/f2653b2277a64b6191902e0cc0872543/users/5ea8755120d54fd397d7ce0f371b0557/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:34.297326 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e36f835-a8ca-427d-8856-4b7a1bd01d1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2653b2277a64b6191902e0cc0872543, user_id=5ea8755120d54fd397d7ce0f371b0557, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:34.300631 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e36f835-a8ca-427d-8856-4b7a1bd01d1d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:34.305491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a597935e-aa71-462f-ac95-a8be1c628506 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70c3d673547c4f45945a97eb4921bac2'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:34.306695 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a597935e-aa71-462f-ac95-a8be1c628506 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70c3d673547c4f45945a97eb4921bac2'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:34.314188 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 826/1632] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:34 2022] PUT /identity/v3/projects/f2653b2277a64b6191902e0cc0872543/users/5ea8755120d54fd397d7ce0f371b0557/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:05:34.314659 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a597935e-aa71-462f-ac95-a8be1c628506 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70c3d673547c4f45945a97eb4921bac2'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:34.324556 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089982f-8128-4eaf-8a5f-afa81feb0294 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:34.324927 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089982f-8128-4eaf-8a5f-afa81feb0294 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:34.325192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089982f-8128-4eaf-8a5f-afa81feb0294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:34.329006 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a597935e-aa71-462f-ac95-a8be1c628506 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70c3d673547c4f45945a97eb4921bac2'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:34.337166 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a597935e-aa71-462f-ac95-a8be1c628506 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70c3d673547c4f45945a97eb4921bac2'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:34.344163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 807/1633] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:34 2022] DELETE /identity/v3/users/70c3d673547c4f45945a97eb4921bac2 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:05:34.357571 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-9089982f-8128-4eaf-8a5f-afa81feb0294 None None] MFA Rules not processed for user `5ea8755120d54fd397d7ce0f371b0557`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:34.358838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9089982f-8128-4eaf-8a5f-afa81feb0294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:34.420264 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:34.445190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 827/1634] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:34 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:34.456832 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597935e-aa71-462f-ac95-a8be1c628506 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:34.459267 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a597935e-aa71-462f-ac95-a8be1c628506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:34.552453 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597935e-aa71-462f-ac95-a8be1c628506 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:34.560709 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f89498-e201-4e50-84eb-799f5e97b4a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:34.561032 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f89498-e201-4e50-84eb-799f5e97b4a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:34.561315 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f89498-e201-4e50-84eb-799f5e97b4a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:34.561589 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f89498-e201-4e50-84eb-799f5e97b4a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:34.572187 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f89498-e201-4e50-84eb-799f5e97b4a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:34.573664 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f89498-e201-4e50-84eb-799f5e97b4a4 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:34.585192 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 808/1635] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:34 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:34.664056 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089982f-8128-4eaf-8a5f-afa81feb0294 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:34.666357 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9089982f-8128-4eaf-8a5f-afa81feb0294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:34.740351 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089982f-8128-4eaf-8a5f-afa81feb0294 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:34.746827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8894ee-2f68-41c9-8c27-7840faaaae43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:34.747467 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8894ee-2f68-41c9-8c27-7840faaaae43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:34.747744 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8894ee-2f68-41c9-8c27-7840faaaae43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:34.748014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8894ee-2f68-41c9-8c27-7840faaaae43 admin admin] PATH_INFO: `/v3/projects/eaba857bf28e465289cdb68ec1cb2cc5` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:34.749303 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8894ee-2f68-41c9-8c27-7840faaaae43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaba857bf28e465289cdb68ec1cb2cc5)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:34.751348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8894ee-2f68-41c9-8c27-7840faaaae43 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:34.805934 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6f8894ee-2f68-41c9-8c27-7840faaaae43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaba857bf28e465289cdb68ec1cb2cc5', 'request_id': 'req-6f8894ee-2f68-41c9-8c27-7840faaaae43'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:34.811632 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6f8894ee-2f68-41c9-8c27-7840faaaae43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaba857bf28e465289cdb68ec1cb2cc5', 'request_id': 'req-6f8894ee-2f68-41c9-8c27-7840faaaae43'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:34.820531 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6f8894ee-2f68-41c9-8c27-7840faaaae43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaba857bf28e465289cdb68ec1cb2cc5', 'request_id': 'req-6f8894ee-2f68-41c9-8c27-7840faaaae43'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:34.833925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6f8894ee-2f68-41c9-8c27-7840faaaae43 admin admin] The token cache is being invalidate because project eaba857bf28e465289cdb68ec1cb2cc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:34.834197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6f8894ee-2f68-41c9-8c27-7840faaaae43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:34.834919 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6f8894ee-2f68-41c9-8c27-7840faaaae43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:34.836679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 828/1636] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:34 2022] DELETE /identity/v3/projects/eaba857bf28e465289cdb68ec1cb2cc5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:34.841229 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f89498-e201-4e50-84eb-799f5e97b4a4 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:34.852060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f89498-e201-4e50-84eb-799f5e97b4a4 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:34.877164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:34.914725 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85f317-eee8-43a3-a543-85fd6cf470b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:34.915242 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85f317-eee8-43a3-a543-85fd6cf470b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:34.915584 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85f317-eee8-43a3-a543-85fd6cf470b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:34.915875 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85f317-eee8-43a3-a543-85fd6cf470b0 admin admin] PATH_INFO: `/v3/users/5508663f152442ca84f0e7f4e38efc72` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:34.917030 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85f317-eee8-43a3-a543-85fd6cf470b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5508663f152442ca84f0e7f4e38efc72)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:34.919058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85f317-eee8-43a3-a543-85fd6cf470b0 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:34.970083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3b85f317-eee8-43a3-a543-85fd6cf470b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5508663f152442ca84f0e7f4e38efc72'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:34.971762 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3b85f317-eee8-43a3-a543-85fd6cf470b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5508663f152442ca84f0e7f4e38efc72'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:34.984846 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3b85f317-eee8-43a3-a543-85fd6cf470b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5508663f152442ca84f0e7f4e38efc72'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:34.993761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3b85f317-eee8-43a3-a543-85fd6cf470b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5508663f152442ca84f0e7f4e38efc72'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:35.000877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3b85f317-eee8-43a3-a543-85fd6cf470b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5508663f152442ca84f0e7f4e38efc72'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:35.007243 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 809/1637] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:34 2022] DELETE /identity/v3/users/5508663f152442ca84f0e7f4e38efc72 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:35.240378 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8894ee-2f68-41c9-8c27-7840faaaae43 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:35.241900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6f8894ee-2f68-41c9-8c27-7840faaaae43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:35.255013 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8894ee-2f68-41c9-8c27-7840faaaae43 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:35.301946 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85f317-eee8-43a3-a543-85fd6cf470b0 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:35.311431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3b85f317-eee8-43a3-a543-85fd6cf470b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:35.361057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e70bb-159e-4e71-8c95-be2d9bba1335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:35.361698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e70bb-159e-4e71-8c95-be2d9bba1335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:35.361996 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e70bb-159e-4e71-8c95-be2d9bba1335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:35.362277 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e70bb-159e-4e71-8c95-be2d9bba1335 admin admin] PATH_INFO: `/v3/users/cc9d8d424c9048fdacac71bc279467fa` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:35.363427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e70bb-159e-4e71-8c95-be2d9bba1335 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc9d8d424c9048fdacac71bc279467fa)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:35.365390 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e70bb-159e-4e71-8c95-be2d9bba1335 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:35.396569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:35.410205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85f317-eee8-43a3-a543-85fd6cf470b0 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:35.415275 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91707957-47cf-42db-8e71-d0ac3650e0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:35.415928 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91707957-47cf-42db-8e71-d0ac3650e0ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:35.416204 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91707957-47cf-42db-8e71-d0ac3650e0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:35.416472 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91707957-47cf-42db-8e71-d0ac3650e0ae admin admin] PATH_INFO: `/v3/projects/7e1010bbafd34c7296d6b3416439815c` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:35.417628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91707957-47cf-42db-8e71-d0ac3650e0ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e1010bbafd34c7296d6b3416439815c)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:35.419538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91707957-47cf-42db-8e71-d0ac3650e0ae admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:35.428034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b38e70bb-159e-4e71-8c95-be2d9bba1335 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc9d8d424c9048fdacac71bc279467fa'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:35.428862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b38e70bb-159e-4e71-8c95-be2d9bba1335 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc9d8d424c9048fdacac71bc279467fa'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:35.450143 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b38e70bb-159e-4e71-8c95-be2d9bba1335 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc9d8d424c9048fdacac71bc279467fa'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:35.472535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b38e70bb-159e-4e71-8c95-be2d9bba1335 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc9d8d424c9048fdacac71bc279467fa'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:35.473243 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b38e70bb-159e-4e71-8c95-be2d9bba1335 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc9d8d424c9048fdacac71bc279467fa'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:35.482778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:35.485936 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-91707957-47cf-42db-8e71-d0ac3650e0ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e1010bbafd34c7296d6b3416439815c', 'request_id': 'req-91707957-47cf-42db-8e71-d0ac3650e0ae'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:35.493715 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 829/1638] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:35 2022] DELETE /identity/v3/users/cc9d8d424c9048fdacac71bc279467fa => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:35.497559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-91707957-47cf-42db-8e71-d0ac3650e0ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e1010bbafd34c7296d6b3416439815c', 'request_id': 'req-91707957-47cf-42db-8e71-d0ac3650e0ae'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:35.502050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1927a47-8008-42d8-95ed-5175edd9dee6 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:35.502524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1927a47-8008-42d8-95ed-5175edd9dee6 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:35.503074 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1927a47-8008-42d8-95ed-5175edd9dee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:35.506979 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-91707957-47cf-42db-8e71-d0ac3650e0ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e1010bbafd34c7296d6b3416439815c', 'request_id': 'req-91707957-47cf-42db-8e71-d0ac3650e0ae'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:35.521923 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-91707957-47cf-42db-8e71-d0ac3650e0ae admin admin] The token cache is being invalidate because project 7e1010bbafd34c7296d6b3416439815c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:35.522288 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-91707957-47cf-42db-8e71-d0ac3650e0ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:35.523037 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-91707957-47cf-42db-8e71-d0ac3650e0ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:35.529051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 810/1639] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:35 2022] DELETE /identity/v3/projects/7e1010bbafd34c7296d6b3416439815c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:35.530213 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-b1927a47-8008-42d8-95ed-5175edd9dee6 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:35.530755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4636d19-3f2b-4911-8e7f-c80864dfe023 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:35.531042 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4636d19-3f2b-4911-8e7f-c80864dfe023 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:35.531314 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4636d19-3f2b-4911-8e7f-c80864dfe023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:35.531751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b1927a47-8008-42d8-95ed-5175edd9dee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:35.559016 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-e4636d19-3f2b-4911-8e7f-c80864dfe023 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:35.560290 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e4636d19-3f2b-4911-8e7f-c80864dfe023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:35.600195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 830/1640] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:35.611325 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1927a47-8008-42d8-95ed-5175edd9dee6 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:35.614696 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b1927a47-8008-42d8-95ed-5175edd9dee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:35.623802 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 811/1641] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:05:35.632546 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9ab9c-b456-4be5-88d1-84633bd84f7e None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:35.632935 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9ab9c-b456-4be5-88d1-84633bd84f7e None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:35.633276 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9ab9c-b456-4be5-88d1-84633bd84f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:35.659171 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-82f9ab9c-b456-4be5-88d1-84633bd84f7e None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:35.664967 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-82f9ab9c-b456-4be5-88d1-84633bd84f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:35.677837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 812/1642] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:35.678510 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9ab9c-b456-4be5-88d1-84633bd84f7e None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:35.699307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1927a47-8008-42d8-95ed-5175edd9dee6 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:35.707155 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d4584-4dbd-4708-8918-546c17c476ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:35.707952 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d4584-4dbd-4708-8918-546c17c476ad service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:35.708323 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d4584-4dbd-4708-8918-546c17c476ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:35.708675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d4584-4dbd-4708-8918-546c17c476ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:35.709316 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9ab9c-b456-4be5-88d1-84633bd84f7e None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:35.722450 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d4584-4dbd-4708-8918-546c17c476ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:35.724154 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d4584-4dbd-4708-8918-546c17c476ad service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:35.728091 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991a62b-dc2b-4ff9-a2ce-362afdcf2acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:35.728744 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991a62b-dc2b-4ff9-a2ce-362afdcf2acd admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:35.729021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991a62b-dc2b-4ff9-a2ce-362afdcf2acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:35.729292 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991a62b-dc2b-4ff9-a2ce-362afdcf2acd admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:35.729945 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e991a62b-dc2b-4ff9-a2ce-362afdcf2acd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:05:35.730249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e991a62b-dc2b-4ff9-a2ce-362afdcf2acd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:35.731904 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e991a62b-dc2b-4ff9-a2ce-362afdcf2acd admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:35.747692 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 813/1644] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 06:05:35.750955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 831/1644] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:05:35.756310 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991a62b-dc2b-4ff9-a2ce-362afdcf2acd admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:35.770956 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991a62b-dc2b-4ff9-a2ce-362afdcf2acd admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:35.776869 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f4a76-f997-42d5-bf82-18427bbd098a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:35.777666 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f4a76-f997-42d5-bf82-18427bbd098a admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:35.777942 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f4a76-f997-42d5-bf82-18427bbd098a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:35.778213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f4a76-f997-42d5-bf82-18427bbd098a admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:35.778851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f4a76-f997-42d5-bf82-18427bbd098a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:35.780779 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f4a76-f997-42d5-bf82-18427bbd098a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:35.820958 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 814/1645] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:35 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:05:35.823345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d4584-4dbd-4708-8918-546c17c476ad service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:35.836680 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d4584-4dbd-4708-8918-546c17c476ad service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:35.908322 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:35.932392 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ca2ac-7e59-4a5e-884a-ecad3cec9903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:35.933476 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ca2ac-7e59-4a5e-884a-ecad3cec9903 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:35.933476 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ca2ac-7e59-4a5e-884a-ecad3cec9903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:35.933765 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ca2ac-7e59-4a5e-884a-ecad3cec9903 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:35.934706 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ca2ac-7e59-4a5e-884a-ecad3cec9903 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:35.936882 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ca2ac-7e59-4a5e-884a-ecad3cec9903 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:35.963630 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 832/1646] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:35 2022] POST /identity/v3/users => generated 292 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:35.964603 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f4a76-f997-42d5-bf82-18427bbd098a admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:35.975714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f4a76-f997-42d5-bf82-18427bbd098a admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:35.981079 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3ac41-206e-4bc7-becc-30c84abf8500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:35.981791 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3ac41-206e-4bc7-becc-30c84abf8500 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:35.982072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3ac41-206e-4bc7-becc-30c84abf8500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:35.982473 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3ac41-206e-4bc7-becc-30c84abf8500 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:35.983209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3ac41-206e-4bc7-becc-30c84abf8500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:35.984845 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3ac41-206e-4bc7-becc-30c84abf8500 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:35.996360 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 815/1647] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:36.000231 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ca2ac-7e59-4a5e-884a-ecad3cec9903 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:36.013980 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ca2ac-7e59-4a5e-884a-ecad3cec9903 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:36.022545 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519aaee-0717-4161-b398-5adb0353277e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:36.023278 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519aaee-0717-4161-b398-5adb0353277e admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:36.023589 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519aaee-0717-4161-b398-5adb0353277e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:36.023835 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519aaee-0717-4161-b398-5adb0353277e admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:36.024518 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2519aaee-0717-4161-b398-5adb0353277e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:36.026677 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2519aaee-0717-4161-b398-5adb0353277e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:36.039712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 833/1648] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:05:36.043887 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3ac41-206e-4bc7-becc-30c84abf8500 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:36.063331 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3ac41-206e-4bc7-becc-30c84abf8500 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:36.069322 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7278b459-f4e3-4a1d-93ee-c4b7c7c3f635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:36.072331 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7278b459-f4e3-4a1d-93ee-c4b7c7c3f635 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:36.072331 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7278b459-f4e3-4a1d-93ee-c4b7c7c3f635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:36.072331 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7278b459-f4e3-4a1d-93ee-c4b7c7c3f635 admin admin] PATH_INFO: `/v3/projects/3de3d03dde2d4ef6a8b925a0a5391d0b/users/7935a2a80f054f699983e5e2f3e64a28/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:36.073565 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519aaee-0717-4161-b398-5adb0353277e admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:36.076379 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2519aaee-0717-4161-b398-5adb0353277e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:36.092161 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519aaee-0717-4161-b398-5adb0353277e admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:36.099413 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7278b459-f4e3-4a1d-93ee-c4b7c7c3f635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3de3d03dde2d4ef6a8b925a0a5391d0b, user_id=7935a2a80f054f699983e5e2f3e64a28, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:36.100931 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae17f0c-915a-4e08-911d-7c989707e537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:36.101620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae17f0c-915a-4e08-911d-7c989707e537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:36.101905 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae17f0c-915a-4e08-911d-7c989707e537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:36.102115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7278b459-f4e3-4a1d-93ee-c4b7c7c3f635 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:36.102218 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae17f0c-915a-4e08-911d-7c989707e537 admin admin] PATH_INFO: `/v3/projects/ae45d83432b34e2aa9c24c60260b8ea1` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:36.104345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae17f0c-915a-4e08-911d-7c989707e537 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae45d83432b34e2aa9c24c60260b8ea1)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:36.106265 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae17f0c-915a-4e08-911d-7c989707e537 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:36.123118 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 816/1649] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:36 2022] PUT /identity/v3/projects/3de3d03dde2d4ef6a8b925a0a5391d0b/users/7935a2a80f054f699983e5e2f3e64a28/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:36.124072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7278b459-f4e3-4a1d-93ee-c4b7c7c3f635 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:36.126096 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7278b459-f4e3-4a1d-93ee-c4b7c7c3f635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:36.162725 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-aae17f0c-915a-4e08-911d-7c989707e537 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae45d83432b34e2aa9c24c60260b8ea1', 'request_id': 'req-aae17f0c-915a-4e08-911d-7c989707e537'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:36.168408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-aae17f0c-915a-4e08-911d-7c989707e537 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae45d83432b34e2aa9c24c60260b8ea1', 'request_id': 'req-aae17f0c-915a-4e08-911d-7c989707e537'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:36.175760 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-aae17f0c-915a-4e08-911d-7c989707e537 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae45d83432b34e2aa9c24c60260b8ea1', 'request_id': 'req-aae17f0c-915a-4e08-911d-7c989707e537'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:36.188304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-aae17f0c-915a-4e08-911d-7c989707e537 admin admin] The token cache is being invalidate because project ae45d83432b34e2aa9c24c60260b8ea1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:36.188612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-aae17f0c-915a-4e08-911d-7c989707e537 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:36.189838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-aae17f0c-915a-4e08-911d-7c989707e537 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:36.192944 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 834/1650] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:36 2022] DELETE /identity/v3/projects/ae45d83432b34e2aa9c24c60260b8ea1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:05:36.195494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7278b459-f4e3-4a1d-93ee-c4b7c7c3f635 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:36.198079 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8981bae-5baf-46cc-843d-04174a9e176e None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:36.198364 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8981bae-5baf-46cc-843d-04174a9e176e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:36.198662 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8981bae-5baf-46cc-843d-04174a9e176e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:36.199563 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:36.220360 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-f8981bae-5baf-46cc-843d-04174a9e176e None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:36.221883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f8981bae-5baf-46cc-843d-04174a9e176e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:36.274396 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914af40-5fa6-46d2-a27f-55358d285523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:36.274972 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914af40-5fa6-46d2-a27f-55358d285523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:36.275289 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914af40-5fa6-46d2-a27f-55358d285523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:36.275518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914af40-5fa6-46d2-a27f-55358d285523 admin admin] PATH_INFO: `/v3/users/aa1775c61bb94268ab327641b731de65` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:36.276721 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914af40-5fa6-46d2-a27f-55358d285523 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa1775c61bb94268ab327641b731de65)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:36.278855 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914af40-5fa6-46d2-a27f-55358d285523 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:36.303185 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 835/1651] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:36.316361 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8981bae-5baf-46cc-843d-04174a9e176e None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:36.319826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f8981bae-5baf-46cc-843d-04174a9e176e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:36.354939 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1914af40-5fa6-46d2-a27f-55358d285523 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa1775c61bb94268ab327641b731de65'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:36.355776 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1914af40-5fa6-46d2-a27f-55358d285523 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa1775c61bb94268ab327641b731de65'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:36.367854 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1914af40-5fa6-46d2-a27f-55358d285523 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa1775c61bb94268ab327641b731de65'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:36.385755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1914af40-5fa6-46d2-a27f-55358d285523 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa1775c61bb94268ab327641b731de65'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:36.396057 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1914af40-5fa6-46d2-a27f-55358d285523 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa1775c61bb94268ab327641b731de65'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:36.397782 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:36.402633 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 817/1652] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:36 2022] DELETE /identity/v3/users/aa1775c61bb94268ab327641b731de65 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:36.404447 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8981bae-5baf-46cc-843d-04174a9e176e None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:36.407069 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab63965-642d-466c-818d-ffec2138ee0a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:36.407372 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab63965-642d-466c-818d-ffec2138ee0a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:36.407727 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab63965-642d-466c-818d-ffec2138ee0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:36.426982 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-aab63965-642d-466c-818d-ffec2138ee0a None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:36.428184 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-aab63965-642d-466c-818d-ffec2138ee0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:36.459384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e266f7-fe15-4b29-b8f8-3606a3ecbb27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:36.459998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e266f7-fe15-4b29-b8f8-3606a3ecbb27 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:36.460269 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e266f7-fe15-4b29-b8f8-3606a3ecbb27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:36.460537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e266f7-fe15-4b29-b8f8-3606a3ecbb27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:36.462297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a3e266f7-fe15-4b29-b8f8-3606a3ecbb27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:36.489129 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 818/1653] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:05:36.502223 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab63965-642d-466c-818d-ffec2138ee0a None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:36.504279 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-aab63965-642d-466c-818d-ffec2138ee0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:36.519278 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab63965-642d-466c-818d-ffec2138ee0a None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:36.523944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab640967-d2a1-4069-a53b-ef2d9eafb1f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:36.524838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab640967-d2a1-4069-a53b-ef2d9eafb1f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:36.525056 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab640967-d2a1-4069-a53b-ef2d9eafb1f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:36.525867 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab640967-d2a1-4069-a53b-ef2d9eafb1f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:36.527670 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e266f7-fe15-4b29-b8f8-3606a3ecbb27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:36.529920 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e266f7-fe15-4b29-b8f8-3606a3ecbb27 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:36.536210 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab640967-d2a1-4069-a53b-ef2d9eafb1f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:36.537731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab640967-d2a1-4069-a53b-ef2d9eafb1f6 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:36.550098 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 819/1654] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:05:36.552822 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 836/1655] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:05:36.948250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab640967-d2a1-4069-a53b-ef2d9eafb1f6 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:36.949939 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ab640967-d2a1-4069-a53b-ef2d9eafb1f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:36.967206 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab640967-d2a1-4069-a53b-ef2d9eafb1f6 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:36.973541 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef63d4-68bb-4c11-b15a-b51f1b8707d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:36.974207 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef63d4-68bb-4c11-b15a-b51f1b8707d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:36.974487 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef63d4-68bb-4c11-b15a-b51f1b8707d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:36.974755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef63d4-68bb-4c11-b15a-b51f1b8707d5 admin admin] PATH_INFO: `/v3/projects/249dc6c053374604b414f32f290f6b28` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:36.975874 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef63d4-68bb-4c11-b15a-b51f1b8707d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=249dc6c053374604b414f32f290f6b28)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:36.978830 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef63d4-68bb-4c11-b15a-b51f1b8707d5 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:36.990125 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:37.072459 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-64ef63d4-68bb-4c11-b15a-b51f1b8707d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '249dc6c053374604b414f32f290f6b28', 'request_id': 'req-64ef63d4-68bb-4c11-b15a-b51f1b8707d5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:37.079627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-64ef63d4-68bb-4c11-b15a-b51f1b8707d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '249dc6c053374604b414f32f290f6b28', 'request_id': 'req-64ef63d4-68bb-4c11-b15a-b51f1b8707d5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:37.089967 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-64ef63d4-68bb-4c11-b15a-b51f1b8707d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '249dc6c053374604b414f32f290f6b28', 'request_id': 'req-64ef63d4-68bb-4c11-b15a-b51f1b8707d5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:37.130838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-64ef63d4-68bb-4c11-b15a-b51f1b8707d5 admin admin] The token cache is being invalidate because project 249dc6c053374604b414f32f290f6b28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:37.130838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-64ef63d4-68bb-4c11-b15a-b51f1b8707d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:37.130838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-64ef63d4-68bb-4c11-b15a-b51f1b8707d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:37.130838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 820/1656] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:36 2022] DELETE /identity/v3/projects/249dc6c053374604b414f32f290f6b28 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:37.132068 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e266f7-fe15-4b29-b8f8-3606a3ecbb27 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:37.137210 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e266f7-fe15-4b29-b8f8-3606a3ecbb27 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:37.196601 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d570d615-8b77-4b94-a029-4a0211e739dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:37.197280 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570d615-8b77-4b94-a029-4a0211e739dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:37.197561 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570d615-8b77-4b94-a029-4a0211e739dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:37.197835 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570d615-8b77-4b94-a029-4a0211e739dc admin admin] PATH_INFO: `/v3/users/5ea8755120d54fd397d7ce0f371b0557` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:37.198959 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d570d615-8b77-4b94-a029-4a0211e739dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ea8755120d54fd397d7ce0f371b0557)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:37.200904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d570d615-8b77-4b94-a029-4a0211e739dc admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:37.253798 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d570d615-8b77-4b94-a029-4a0211e739dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ea8755120d54fd397d7ce0f371b0557'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:37.254477 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d570d615-8b77-4b94-a029-4a0211e739dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ea8755120d54fd397d7ce0f371b0557'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:37.263059 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d570d615-8b77-4b94-a029-4a0211e739dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ea8755120d54fd397d7ce0f371b0557'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:37.264432 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:37.280867 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d570d615-8b77-4b94-a029-4a0211e739dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ea8755120d54fd397d7ce0f371b0557'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:37.281838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d570d615-8b77-4b94-a029-4a0211e739dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ea8755120d54fd397d7ce0f371b0557'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:37.294994 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 837/1657] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:37 2022] DELETE /identity/v3/users/5ea8755120d54fd397d7ce0f371b0557 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:37.425516 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b98d5b-ec1d-4ae0-a32a-fc9f580300b5 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:37.425808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b98d5b-ec1d-4ae0-a32a-fc9f580300b5 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:37.426070 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b98d5b-ec1d-4ae0-a32a-fc9f580300b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:37.457347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-69b98d5b-ec1d-4ae0-a32a-fc9f580300b5 None None] MFA Rules not processed for user `7935a2a80f054f699983e5e2f3e64a28`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:37.458550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-69b98d5b-ec1d-4ae0-a32a-fc9f580300b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:37.545162 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 821/1658] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:37.556671 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d570d615-8b77-4b94-a029-4a0211e739dc admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:37.559039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d570d615-8b77-4b94-a029-4a0211e739dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:37.650392 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d570d615-8b77-4b94-a029-4a0211e739dc admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:37.655624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05854c70-ac66-4ddd-989b-b0414fd9b483 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:37.656036 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b98d5b-ec1d-4ae0-a32a-fc9f580300b5 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:37.656343 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05854c70-ac66-4ddd-989b-b0414fd9b483 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:37.656631 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05854c70-ac66-4ddd-989b-b0414fd9b483 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:37.656941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05854c70-ac66-4ddd-989b-b0414fd9b483 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:37.657626 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-69b98d5b-ec1d-4ae0-a32a-fc9f580300b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:37.676382 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05854c70-ac66-4ddd-989b-b0414fd9b483 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:37.677907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05854c70-ac66-4ddd-989b-b0414fd9b483 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:37.697418 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 838/1659] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:37.744763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b98d5b-ec1d-4ae0-a32a-fc9f580300b5 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:37.750721 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dccd993-c25f-4994-b1b9-987034b21190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:37.751380 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccd993-c25f-4994-b1b9-987034b21190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:37.751662 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccd993-c25f-4994-b1b9-987034b21190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:37.751933 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccd993-c25f-4994-b1b9-987034b21190 admin admin] PATH_INFO: `/v3/projects/f2653b2277a64b6191902e0cc0872543` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:37.753745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dccd993-c25f-4994-b1b9-987034b21190 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2653b2277a64b6191902e0cc0872543)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:37.755693 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dccd993-c25f-4994-b1b9-987034b21190 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:37.794865 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:37.816996 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5dccd993-c25f-4994-b1b9-987034b21190 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2653b2277a64b6191902e0cc0872543', 'request_id': 'req-5dccd993-c25f-4994-b1b9-987034b21190'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:37.824593 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5dccd993-c25f-4994-b1b9-987034b21190 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2653b2277a64b6191902e0cc0872543', 'request_id': 'req-5dccd993-c25f-4994-b1b9-987034b21190'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:37.831955 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5dccd993-c25f-4994-b1b9-987034b21190 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2653b2277a64b6191902e0cc0872543', 'request_id': 'req-5dccd993-c25f-4994-b1b9-987034b21190'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:37.844928 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5dccd993-c25f-4994-b1b9-987034b21190 admin admin] The token cache is being invalidate because project f2653b2277a64b6191902e0cc0872543 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:37.845346 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5dccd993-c25f-4994-b1b9-987034b21190 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:37.847258 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5dccd993-c25f-4994-b1b9-987034b21190 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:37.848953 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 822/1660] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:37 2022] DELETE /identity/v3/projects/f2653b2277a64b6191902e0cc0872543 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:37.854550 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23071884-d551-4b37-b88d-a77294421ab0 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:37.854903 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23071884-d551-4b37-b88d-a77294421ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:37.855274 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23071884-d551-4b37-b88d-a77294421ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:37.878347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-23071884-d551-4b37-b88d-a77294421ab0 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:37.879679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-23071884-d551-4b37-b88d-a77294421ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:37.979243 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 839/1661] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:37.998323 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01de6b7-ae12-471e-9da8-9fc1b6b34a5b None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:37.998798 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01de6b7-ae12-471e-9da8-9fc1b6b34a5b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:37.998990 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01de6b7-ae12-471e-9da8-9fc1b6b34a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:38.023864 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f01de6b7-ae12-471e-9da8-9fc1b6b34a5b None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:38.025848 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f01de6b7-ae12-471e-9da8-9fc1b6b34a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:38.041325 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 823/1662] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:38.046227 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23071884-d551-4b37-b88d-a77294421ab0 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:38.058886 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23071884-d551-4b37-b88d-a77294421ab0 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:38.063984 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b10252-be3f-43eb-a8a1-4802a189d349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:38.064603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b10252-be3f-43eb-a8a1-4802a189d349 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:38.065426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b10252-be3f-43eb-a8a1-4802a189d349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:38.065886 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b10252-be3f-43eb-a8a1-4802a189d349 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:38.066622 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b10252-be3f-43eb-a8a1-4802a189d349 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:05:38.067268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b10252-be3f-43eb-a8a1-4802a189d349 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:38.068779 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b10252-be3f-43eb-a8a1-4802a189d349 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:38.082707 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 840/1663] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:05:38.087023 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01de6b7-ae12-471e-9da8-9fc1b6b34a5b None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:38.097100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01de6b7-ae12-471e-9da8-9fc1b6b34a5b None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:38.101866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edf50c-6339-49c4-8ff8-0eb4b5923d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:38.102608 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edf50c-6339-49c4-8ff8-0eb4b5923d8b admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:38.102776 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edf50c-6339-49c4-8ff8-0eb4b5923d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:38.103018 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edf50c-6339-49c4-8ff8-0eb4b5923d8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:38.103691 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edf50c-6339-49c4-8ff8-0eb4b5923d8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:38.105632 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edf50c-6339-49c4-8ff8-0eb4b5923d8b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:38.134730 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 824/1664] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:38 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:05:38.138595 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b10252-be3f-43eb-a8a1-4802a189d349 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:38.149230 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b10252-be3f-43eb-a8a1-4802a189d349 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:38.208594 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c8c7a-c53f-40e6-9d1a-34e58a7eee0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:38.209389 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c8c7a-c53f-40e6-9d1a-34e58a7eee0b admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:38.209670 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c8c7a-c53f-40e6-9d1a-34e58a7eee0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:38.209943 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c8c7a-c53f-40e6-9d1a-34e58a7eee0b admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:38.210571 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c8c7a-c53f-40e6-9d1a-34e58a7eee0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:38.212489 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c8c7a-c53f-40e6-9d1a-34e58a7eee0b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:38.221973 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:38.240553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 841/1665] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:38 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:38.241855 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edf50c-6339-49c4-8ff8-0eb4b5923d8b admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:38.253112 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edf50c-6339-49c4-8ff8-0eb4b5923d8b admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:38.257959 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895701b2-0183-4e9e-b77a-1d534fedc9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:38.258561 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895701b2-0183-4e9e-b77a-1d534fedc9fd admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:38.259114 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895701b2-0183-4e9e-b77a-1d534fedc9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:38.259390 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895701b2-0183-4e9e-b77a-1d534fedc9fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:38.260020 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895701b2-0183-4e9e-b77a-1d534fedc9fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:38.261649 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895701b2-0183-4e9e-b77a-1d534fedc9fd admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:38.272166 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 825/1666] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:38.275873 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c8c7a-c53f-40e6-9d1a-34e58a7eee0b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:38.286828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c8c7a-c53f-40e6-9d1a-34e58a7eee0b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:38.291572 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc7d08-34cb-412e-9c17-b1c855f5d201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:38.292176 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc7d08-34cb-412e-9c17-b1c855f5d201 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:38.292452 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc7d08-34cb-412e-9c17-b1c855f5d201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:38.292773 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc7d08-34cb-412e-9c17-b1c855f5d201 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:38.293401 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc7d08-34cb-412e-9c17-b1c855f5d201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:38.295053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc7d08-34cb-412e-9c17-b1c855f5d201 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:38.315709 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 842/1667] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:38.319980 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895701b2-0183-4e9e-b77a-1d534fedc9fd admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:38.329939 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895701b2-0183-4e9e-b77a-1d534fedc9fd admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:38.334951 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586277c-99d2-458e-9f7f-008b179e8ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:38.335603 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586277c-99d2-458e-9f7f-008b179e8ef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:38.335878 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586277c-99d2-458e-9f7f-008b179e8ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:38.336146 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586277c-99d2-458e-9f7f-008b179e8ef4 admin admin] PATH_INFO: `/v3/projects/5cb5e0eff6a34c85aa6bd72ca42a1f5f/users/514094181b764d5380b2d9dbd45106a5/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:38.350651 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586277c-99d2-458e-9f7f-008b179e8ef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cb5e0eff6a34c85aa6bd72ca42a1f5f, user_id=514094181b764d5380b2d9dbd45106a5, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:38.353386 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586277c-99d2-458e-9f7f-008b179e8ef4 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:38.366445 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 826/1668] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:38 2022] PUT /identity/v3/projects/5cb5e0eff6a34c85aa6bd72ca42a1f5f/users/514094181b764d5380b2d9dbd45106a5/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:05:38.377133 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc3739-dcdd-4531-bef3-62500604d297 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:38.377975 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc3739-dcdd-4531-bef3-62500604d297 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:38.378249 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc3739-dcdd-4531-bef3-62500604d297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:38.405547 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-7dbc3739-dcdd-4531-bef3-62500604d297 None None] MFA Rules not processed for user `514094181b764d5380b2d9dbd45106a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:38.406753 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7dbc3739-dcdd-4531-bef3-62500604d297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:38.470384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 843/1669] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:38.474909 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586277c-99d2-458e-9f7f-008b179e8ef4 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:38.536052 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586277c-99d2-458e-9f7f-008b179e8ef4 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:38.540618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fceedc8-13a5-4bc5-be7d-c20627dc5c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:38.541356 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fceedc8-13a5-4bc5-be7d-c20627dc5c4e admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:38.541648 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fceedc8-13a5-4bc5-be7d-c20627dc5c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:38.541925 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fceedc8-13a5-4bc5-be7d-c20627dc5c4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:38.542558 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fceedc8-13a5-4bc5-be7d-c20627dc5c4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:38.544456 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fceedc8-13a5-4bc5-be7d-c20627dc5c4e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:38.570458 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 827/1670] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:38 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:05:38.574481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc3739-dcdd-4531-bef3-62500604d297 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:38.585603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc3739-dcdd-4531-bef3-62500604d297 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:38.640394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b1dcd-f2f2-4580-b831-863bf75af316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:38.641033 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b1dcd-f2f2-4580-b831-863bf75af316 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:38.641307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b1dcd-f2f2-4580-b831-863bf75af316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:38.641600 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b1dcd-f2f2-4580-b831-863bf75af316 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:38.642228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b1dcd-f2f2-4580-b831-863bf75af316 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:38.644734 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b1dcd-f2f2-4580-b831-863bf75af316 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:38.667148 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 844/1671] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:38 2022] POST /identity/v3/users => generated 298 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:05:38.669318 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fceedc8-13a5-4bc5-be7d-c20627dc5c4e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:38.679491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fceedc8-13a5-4bc5-be7d-c20627dc5c4e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:38.684075 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13680d-5aac-4e26-8533-ec3a3e2a638f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:38.684678 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13680d-5aac-4e26-8533-ec3a3e2a638f admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:38.684987 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13680d-5aac-4e26-8533-ec3a3e2a638f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:38.685263 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13680d-5aac-4e26-8533-ec3a3e2a638f admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:38.685896 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13680d-5aac-4e26-8533-ec3a3e2a638f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:38.687483 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13680d-5aac-4e26-8533-ec3a3e2a638f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:38.699328 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 828/1672] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:38.703406 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b1dcd-f2f2-4580-b831-863bf75af316 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:38.714159 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b1dcd-f2f2-4580-b831-863bf75af316 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:38.718818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb36ed-4f2e-4789-9dbf-22a4fe99714c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:38.719470 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb36ed-4f2e-4789-9dbf-22a4fe99714c admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:38.719750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb36ed-4f2e-4789-9dbf-22a4fe99714c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:38.720022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb36ed-4f2e-4789-9dbf-22a4fe99714c admin admin] PATH_INFO: `/v3/projects/545b9d4350c24091829ec2789535f642/users/2773ad8f9e324d709be39cc7a2068231/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:38.734545 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb36ed-4f2e-4789-9dbf-22a4fe99714c admin admin] RBAC: Authorizing `identity:create_grant(project_id=545b9d4350c24091829ec2789535f642, user_id=2773ad8f9e324d709be39cc7a2068231, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:38.737827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb36ed-4f2e-4789-9dbf-22a4fe99714c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:38.750207 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 845/1673] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:38 2022] PUT /identity/v3/projects/545b9d4350c24091829ec2789535f642/users/2773ad8f9e324d709be39cc7a2068231/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:38.754778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13680d-5aac-4e26-8533-ec3a3e2a638f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:38.765348 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13680d-5aac-4e26-8533-ec3a3e2a638f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:38.789385 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:38.824374 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66800044-a83e-4466-8ccc-104cdbd15aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:38.824951 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66800044-a83e-4466-8ccc-104cdbd15aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:38.825233 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66800044-a83e-4466-8ccc-104cdbd15aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:38.825506 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66800044-a83e-4466-8ccc-104cdbd15aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:38.826136 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66800044-a83e-4466-8ccc-104cdbd15aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:38.827731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66800044-a83e-4466-8ccc-104cdbd15aa6 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:38.837782 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 829/1674] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:38.842144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb36ed-4f2e-4789-9dbf-22a4fe99714c admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:38.852671 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb36ed-4f2e-4789-9dbf-22a4fe99714c admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:38.857231 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb0ca6-aacc-447f-9139-e51a8dddb44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:38.857870 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb0ca6-aacc-447f-9139-e51a8dddb44f admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:38.858141 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb0ca6-aacc-447f-9139-e51a8dddb44f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:38.858410 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb0ca6-aacc-447f-9139-e51a8dddb44f admin admin] PATH_INFO: `/v3/projects/545b9d4350c24091829ec2789535f642/users/2773ad8f9e324d709be39cc7a2068231/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:38.860264 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb0ca6-aacc-447f-9139-e51a8dddb44f admin admin] RBAC: Authorizing `identity:create_grant(project_id=545b9d4350c24091829ec2789535f642, user_id=2773ad8f9e324d709be39cc7a2068231, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:38.863754 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb0ca6-aacc-447f-9139-e51a8dddb44f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:38.873597 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 846/1675] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:38 2022] PUT /identity/v3/projects/545b9d4350c24091829ec2789535f642/users/2773ad8f9e324d709be39cc7a2068231/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:38.884109 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedf850-e2d8-4fd4-9269-c6fb33ffb564 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:38.884396 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedf850-e2d8-4fd4-9269-c6fb33ffb564 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:38.884669 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedf850-e2d8-4fd4-9269-c6fb33ffb564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:38.913110 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-dcedf850-e2d8-4fd4-9269-c6fb33ffb564 None None] MFA Rules not processed for user `2773ad8f9e324d709be39cc7a2068231`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:38.914328 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-dcedf850-e2d8-4fd4-9269-c6fb33ffb564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:38.981315 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 830/1676] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:38 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:38.992607 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb0ca6-aacc-447f-9139-e51a8dddb44f admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:38.994323 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-abcb0ca6-aacc-447f-9139-e51a8dddb44f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:39.025240 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:39.057303 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb0ca6-aacc-447f-9139-e51a8dddb44f admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:39.061459 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af219969-3a24-4262-a810-33bab2d1bcb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:39.062059 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af219969-3a24-4262-a810-33bab2d1bcb4 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:39.062332 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af219969-3a24-4262-a810-33bab2d1bcb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:39.062599 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af219969-3a24-4262-a810-33bab2d1bcb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:39.072742 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af219969-3a24-4262-a810-33bab2d1bcb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:39.074159 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af219969-3a24-4262-a810-33bab2d1bcb4 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:39.083806 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 847/1677] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:39.123794 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc4ff6-89cf-4a0d-8c28-3eee0915b2fd None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:39.124296 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc4ff6-89cf-4a0d-8c28-3eee0915b2fd None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:39.124377 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc4ff6-89cf-4a0d-8c28-3eee0915b2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:39.144840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-0dbc4ff6-89cf-4a0d-8c28-3eee0915b2fd None None] MFA Rules not processed for user `7935a2a80f054f699983e5e2f3e64a28`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:39.145962 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0dbc4ff6-89cf-4a0d-8c28-3eee0915b2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:39.195628 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af219969-3a24-4262-a810-33bab2d1bcb4 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:39.204442 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af219969-3a24-4262-a810-33bab2d1bcb4 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:39.209685 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f2f74-ca65-45b0-8418-70a0efbf03d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:39.210924 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f2f74-ca65-45b0-8418-70a0efbf03d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:39.211308 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f2f74-ca65-45b0-8418-70a0efbf03d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:39.211581 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f2f74-ca65-45b0-8418-70a0efbf03d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:39.214784 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 831/1678] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:39.277833 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f2f74-ca65-45b0-8418-70a0efbf03d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:39.279441 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f2f74-ca65-45b0-8418-70a0efbf03d2 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:39.288590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 848/1679] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:05:42.000447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc4ff6-89cf-4a0d-8c28-3eee0915b2fd None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:42.059519 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:42.071635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc4ff6-89cf-4a0d-8c28-3eee0915b2fd None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:42.076143 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d33f7-fff3-45a9-a0c0-877c089fb01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:42.076800 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d33f7-fff3-45a9-a0c0-877c089fb01b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:42.077078 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d33f7-fff3-45a9-a0c0-877c089fb01b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:42.077346 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d33f7-fff3-45a9-a0c0-877c089fb01b admin admin] PATH_INFO: `/v3/users/514094181b764d5380b2d9dbd45106a5` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:42.078485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d33f7-fff3-45a9-a0c0-877c089fb01b admin admin] RBAC: Authorizing `identity:delete_user(user_id=514094181b764d5380b2d9dbd45106a5)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:42.080931 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d33f7-fff3-45a9-a0c0-877c089fb01b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:42.137759 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-192d33f7-fff3-45a9-a0c0-877c089fb01b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '514094181b764d5380b2d9dbd45106a5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:42.138706 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-192d33f7-fff3-45a9-a0c0-877c089fb01b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '514094181b764d5380b2d9dbd45106a5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:42.148015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-192d33f7-fff3-45a9-a0c0-877c089fb01b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '514094181b764d5380b2d9dbd45106a5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:42.172932 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-192d33f7-fff3-45a9-a0c0-877c089fb01b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '514094181b764d5380b2d9dbd45106a5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:42.182145 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-192d33f7-fff3-45a9-a0c0-877c089fb01b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '514094181b764d5380b2d9dbd45106a5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:42.192569 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 832/1680] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:41 2022] DELETE /identity/v3/users/514094181b764d5380b2d9dbd45106a5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:42.194436 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd23cc-35d7-4ed3-b6f2-f74958c86d55 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:42.194941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd23cc-35d7-4ed3-b6f2-f74958c86d55 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:42.195181 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd23cc-35d7-4ed3-b6f2-f74958c86d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:42.216681 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-29cd23cc-35d7-4ed3-b6f2-f74958c86d55 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:42.218137 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-29cd23cc-35d7-4ed3-b6f2-f74958c86d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:42.291827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 849/1681] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:42.307559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d33f7-fff3-45a9-a0c0-877c089fb01b admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:42.311128 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-192d33f7-fff3-45a9-a0c0-877c089fb01b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:42.387646 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d33f7-fff3-45a9-a0c0-877c089fb01b admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:42.392786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8edba0-ae92-4961-80f1-aa1f18c2ae22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:42.393670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8edba0-ae92-4961-80f1-aa1f18c2ae22 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:42.394415 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8edba0-ae92-4961-80f1-aa1f18c2ae22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:42.394896 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8edba0-ae92-4961-80f1-aa1f18c2ae22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:42.408981 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8edba0-ae92-4961-80f1-aa1f18c2ae22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:42.410577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8edba0-ae92-4961-80f1-aa1f18c2ae22 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:42.421490 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 833/1682] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:42.716255 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd23cc-35d7-4ed3-b6f2-f74958c86d55 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:42.717873 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-29cd23cc-35d7-4ed3-b6f2-f74958c86d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:42.732349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd23cc-35d7-4ed3-b6f2-f74958c86d55 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:42.766974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10646197-a61c-4f40-b7b0-d5b75197e08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:42.767630 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10646197-a61c-4f40-b7b0-d5b75197e08d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:42.768741 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10646197-a61c-4f40-b7b0-d5b75197e08d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:42.769340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10646197-a61c-4f40-b7b0-d5b75197e08d admin admin] PATH_INFO: `/v3/projects/5cb5e0eff6a34c85aa6bd72ca42a1f5f` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:42.770520 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10646197-a61c-4f40-b7b0-d5b75197e08d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cb5e0eff6a34c85aa6bd72ca42a1f5f)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:42.772465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10646197-a61c-4f40-b7b0-d5b75197e08d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:42.839516 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-10646197-a61c-4f40-b7b0-d5b75197e08d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cb5e0eff6a34c85aa6bd72ca42a1f5f', 'request_id': 'req-10646197-a61c-4f40-b7b0-d5b75197e08d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:42.845835 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-10646197-a61c-4f40-b7b0-d5b75197e08d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cb5e0eff6a34c85aa6bd72ca42a1f5f', 'request_id': 'req-10646197-a61c-4f40-b7b0-d5b75197e08d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:42.847598 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:42.854987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-10646197-a61c-4f40-b7b0-d5b75197e08d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cb5e0eff6a34c85aa6bd72ca42a1f5f', 'request_id': 'req-10646197-a61c-4f40-b7b0-d5b75197e08d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:42.871001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-10646197-a61c-4f40-b7b0-d5b75197e08d admin admin] The token cache is being invalidate because project 5cb5e0eff6a34c85aa6bd72ca42a1f5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:42.871384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-10646197-a61c-4f40-b7b0-d5b75197e08d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:42.872067 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-10646197-a61c-4f40-b7b0-d5b75197e08d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:42.873933 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 850/1683] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:42 2022] DELETE /identity/v3/projects/5cb5e0eff6a34c85aa6bd72ca42a1f5f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:42.878281 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8edba0-ae92-4961-80f1-aa1f18c2ae22 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:42.889400 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8edba0-ae92-4961-80f1-aa1f18c2ae22 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:43.004676 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c8349-58a8-40e1-a794-bcc7748fae56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:43.006546 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c8349-58a8-40e1-a794-bcc7748fae56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:43.007229 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c8349-58a8-40e1-a794-bcc7748fae56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:43.007859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c8349-58a8-40e1-a794-bcc7748fae56 admin admin] PATH_INFO: `/v3/users/2773ad8f9e324d709be39cc7a2068231` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:43.009701 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c8349-58a8-40e1-a794-bcc7748fae56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2773ad8f9e324d709be39cc7a2068231)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:43.011789 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c8349-58a8-40e1-a794-bcc7748fae56 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:43.042553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:43.114588 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0a2c8349-58a8-40e1-a794-bcc7748fae56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2773ad8f9e324d709be39cc7a2068231'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:43.115880 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0a2c8349-58a8-40e1-a794-bcc7748fae56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2773ad8f9e324d709be39cc7a2068231'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:43.125948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0a2c8349-58a8-40e1-a794-bcc7748fae56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2773ad8f9e324d709be39cc7a2068231'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:43.146908 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0a2c8349-58a8-40e1-a794-bcc7748fae56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2773ad8f9e324d709be39cc7a2068231'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:43.155371 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0a2c8349-58a8-40e1-a794-bcc7748fae56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2773ad8f9e324d709be39cc7a2068231'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:43.166470 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 834/1684] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:42 2022] DELETE /identity/v3/users/2773ad8f9e324d709be39cc7a2068231 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:43.452890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10646197-a61c-4f40-b7b0-d5b75197e08d admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:43.454646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-10646197-a61c-4f40-b7b0-d5b75197e08d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:43.477108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10646197-a61c-4f40-b7b0-d5b75197e08d admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:43.541204 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314aab0e-ecd2-43c7-83f6-80dbfed8f1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:43.541845 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314aab0e-ecd2-43c7-83f6-80dbfed8f1ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:43.542123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314aab0e-ecd2-43c7-83f6-80dbfed8f1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:43.542395 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314aab0e-ecd2-43c7-83f6-80dbfed8f1ad admin admin] PATH_INFO: `/v3/projects/545b9d4350c24091829ec2789535f642` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:43.543517 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314aab0e-ecd2-43c7-83f6-80dbfed8f1ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=545b9d4350c24091829ec2789535f642)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:43.545466 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314aab0e-ecd2-43c7-83f6-80dbfed8f1ad admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:43.595276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-314aab0e-ecd2-43c7-83f6-80dbfed8f1ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '545b9d4350c24091829ec2789535f642', 'request_id': 'req-314aab0e-ecd2-43c7-83f6-80dbfed8f1ad'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:43.601734 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-314aab0e-ecd2-43c7-83f6-80dbfed8f1ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '545b9d4350c24091829ec2789535f642', 'request_id': 'req-314aab0e-ecd2-43c7-83f6-80dbfed8f1ad'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:43.608019 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-314aab0e-ecd2-43c7-83f6-80dbfed8f1ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '545b9d4350c24091829ec2789535f642', 'request_id': 'req-314aab0e-ecd2-43c7-83f6-80dbfed8f1ad'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:43.621554 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-314aab0e-ecd2-43c7-83f6-80dbfed8f1ad admin admin] The token cache is being invalidate because project 545b9d4350c24091829ec2789535f642 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:43.621781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-314aab0e-ecd2-43c7-83f6-80dbfed8f1ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:43.622732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-314aab0e-ecd2-43c7-83f6-80dbfed8f1ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:43.624100 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 851/1685] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:43 2022] DELETE /identity/v3/projects/545b9d4350c24091829ec2789535f642 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:43.628954 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3641d16f-1d5e-428f-8f88-279d05c9b8d5 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:43.629258 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3641d16f-1d5e-428f-8f88-279d05c9b8d5 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:43.629611 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3641d16f-1d5e-428f-8f88-279d05c9b8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:43.652993 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3641d16f-1d5e-428f-8f88-279d05c9b8d5 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:43.654199 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3641d16f-1d5e-428f-8f88-279d05c9b8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:43.712906 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 835/1686] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:43.721611 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059aab3-5fb8-405e-ac59-0213ad7f3a67 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:43.721892 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059aab3-5fb8-405e-ac59-0213ad7f3a67 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:43.722164 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059aab3-5fb8-405e-ac59-0213ad7f3a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:43.742076 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-6059aab3-5fb8-405e-ac59-0213ad7f3a67 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:43.743283 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6059aab3-5fb8-405e-ac59-0213ad7f3a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:43.751254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 852/1687] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:43.756014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3641d16f-1d5e-428f-8f88-279d05c9b8d5 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:43.765815 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3641d16f-1d5e-428f-8f88-279d05c9b8d5 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:43.770336 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462263e-e853-45df-bf07-17629e831448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:43.770929 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462263e-e853-45df-bf07-17629e831448 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:43.771206 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462263e-e853-45df-bf07-17629e831448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:43.771476 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462263e-e853-45df-bf07-17629e831448 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:43.772117 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462263e-e853-45df-bf07-17629e831448 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:05:43.772418 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462263e-e853-45df-bf07-17629e831448 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:43.774048 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462263e-e853-45df-bf07-17629e831448 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:43.785674 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 836/1688] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:05:43.789591 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059aab3-5fb8-405e-ac59-0213ad7f3a67 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:43.799481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059aab3-5fb8-405e-ac59-0213ad7f3a67 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:43.804475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a91f66-8c92-4d0c-b2aa-3c9498cc6fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:43.805107 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a91f66-8c92-4d0c-b2aa-3c9498cc6fb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:43.805384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a91f66-8c92-4d0c-b2aa-3c9498cc6fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:43.805652 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a91f66-8c92-4d0c-b2aa-3c9498cc6fb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:43.807174 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a91f66-8c92-4d0c-b2aa-3c9498cc6fb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:43.809416 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a91f66-8c92-4d0c-b2aa-3c9498cc6fb1 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:43.837283 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 853/1689] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:43 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:43.841490 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462263e-e853-45df-bf07-17629e831448 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:43.852068 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462263e-e853-45df-bf07-17629e831448 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:43.911301 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a6206-c998-4a14-8ede-626e98ca66a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:43.911907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a6206-c998-4a14-8ede-626e98ca66a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:43.912184 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a6206-c998-4a14-8ede-626e98ca66a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:43.912453 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a6206-c998-4a14-8ede-626e98ca66a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:43.913111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a6206-c998-4a14-8ede-626e98ca66a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:43.914999 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a6206-c998-4a14-8ede-626e98ca66a2 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:43.936427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 837/1690] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:43 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:43.941558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a91f66-8c92-4d0c-b2aa-3c9498cc6fb1 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:43.952598 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a91f66-8c92-4d0c-b2aa-3c9498cc6fb1 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:43.957377 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24162d97-0718-41d3-9141-b10cefd05967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:43.957973 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24162d97-0718-41d3-9141-b10cefd05967 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:43.958249 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24162d97-0718-41d3-9141-b10cefd05967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:43.958513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24162d97-0718-41d3-9141-b10cefd05967 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:43.959169 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24162d97-0718-41d3-9141-b10cefd05967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:43.960766 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24162d97-0718-41d3-9141-b10cefd05967 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:43.970601 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 854/1691] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:43.975232 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a6206-c998-4a14-8ede-626e98ca66a2 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:43.985049 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a6206-c998-4a14-8ede-626e98ca66a2 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:43.990489 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2253ac-69ad-4c16-8270-4e496de6fa80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:43.991086 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2253ac-69ad-4c16-8270-4e496de6fa80 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:43.991362 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2253ac-69ad-4c16-8270-4e496de6fa80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:43.991665 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2253ac-69ad-4c16-8270-4e496de6fa80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:43.992295 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2253ac-69ad-4c16-8270-4e496de6fa80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:43.993906 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2253ac-69ad-4c16-8270-4e496de6fa80 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:43.999014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:44.005207 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 838/1692] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:44.009797 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24162d97-0718-41d3-9141-b10cefd05967 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:44.021170 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24162d97-0718-41d3-9141-b10cefd05967 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:44.025932 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b6d393-621c-464a-880d-586925283fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:44.026583 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6d393-621c-464a-880d-586925283fb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:44.026862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6d393-621c-464a-880d-586925283fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:44.027134 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6d393-621c-464a-880d-586925283fb6 admin admin] PATH_INFO: `/v3/projects/7457566282144956bff50037d25737dc/users/e3bd53f6a4d949df8313ad093c0a5987/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:44.042192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b6d393-621c-464a-880d-586925283fb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7457566282144956bff50037d25737dc, user_id=e3bd53f6a4d949df8313ad093c0a5987, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:44.044842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b6d393-621c-464a-880d-586925283fb6 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:44.058262 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 855/1693] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:44 2022] PUT /identity/v3/projects/7457566282144956bff50037d25737dc/users/e3bd53f6a4d949df8313ad093c0a5987/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:05:44.068413 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff2494-6845-4291-8b28-2ff6b13be4a9 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:44.068746 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff2494-6845-4291-8b28-2ff6b13be4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:44.069025 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff2494-6845-4291-8b28-2ff6b13be4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:44.096165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-daff2494-6845-4291-8b28-2ff6b13be4a9 None None] MFA Rules not processed for user `e3bd53f6a4d949df8313ad093c0a5987`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:44.097424 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-daff2494-6845-4291-8b28-2ff6b13be4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:44.159707 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 839/1694] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:44.170817 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b6d393-621c-464a-880d-586925283fb6 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:44.172474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-81b6d393-621c-464a-880d-586925283fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:44.187959 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:44.245374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b6d393-621c-464a-880d-586925283fb6 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:44.249548 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7725955-ef11-4c24-b4c4-3f8a8ce2162c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:44.250249 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7725955-ef11-4c24-b4c4-3f8a8ce2162c service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:44.250591 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7725955-ef11-4c24-b4c4-3f8a8ce2162c service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:44.252999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7725955-ef11-4c24-b4c4-3f8a8ce2162c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:44.265015 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7725955-ef11-4c24-b4c4-3f8a8ce2162c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:44.266369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7725955-ef11-4c24-b4c4-3f8a8ce2162c service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:44.276875 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 856/1695] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:48.461521 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce2629-098b-43fa-8582-d8d3eadebee6 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:48.462003 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce2629-098b-43fa-8582-d8d3eadebee6 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:48.462094 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce2629-098b-43fa-8582-d8d3eadebee6 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:48.463380 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 840/1696] 10.222.0.20 () {54 vars in 921 bytes} [Thu Oct 20 06:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 06:05:48.473743 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e1e72-d836-42f3-a2a1-193286f36474 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:48.474195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e1e72-d836-42f3-a2a1-193286f36474 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:48.474384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e1e72-d836-42f3-a2a1-193286f36474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:48.495328 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-8c0e1e72-d836-42f3-a2a1-193286f36474 None None] MFA Rules not processed for user `cfbaaf32475a4c82a34948602d8f4e35`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:48.496449 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8c0e1e72-d836-42f3-a2a1-193286f36474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:48.546816 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce2629-098b-43fa-8582-d8d3eadebee6 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:48.553149 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-00ce2629-098b-43fa-8582-d8d3eadebee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:48.570539 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 857/1697] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Oct 20 06:05:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:48.581633 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e1e72-d836-42f3-a2a1-193286f36474 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:48.582986 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8c0e1e72-d836-42f3-a2a1-193286f36474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:48.595370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e1e72-d836-42f3-a2a1-193286f36474 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:48.601084 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68867228-8a12-4b65-8558-2c5fb9a90451 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfbaaf32475a4c82a34948602d8f4e35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:48.601679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68867228-8a12-4b65-8558-2c5fb9a90451 service placement] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:48.601952 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68867228-8a12-4b65-8558-2c5fb9a90451 service placement] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:48.602222 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68867228-8a12-4b65-8558-2c5fb9a90451 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:48.610641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68867228-8a12-4b65-8558-2c5fb9a90451 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:48.612863 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68867228-8a12-4b65-8558-2c5fb9a90451 service placement] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:48.623247 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 858/1698] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:05:48.623912 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce2629-098b-43fa-8582-d8d3eadebee6 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:48.629750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b858251-2b59-440f-8bba-705df22c7ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:48.630391 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b858251-2b59-440f-8bba-705df22c7ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:48.630671 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b858251-2b59-440f-8bba-705df22c7ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:48.630942 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b858251-2b59-440f-8bba-705df22c7ace admin admin] PATH_INFO: `/v3/users/c85953d8e7394fb0a6f425ce51420325` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:48.632083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b858251-2b59-440f-8bba-705df22c7ace admin admin] RBAC: Authorizing `identity:delete_user(user_id=c85953d8e7394fb0a6f425ce51420325)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:48.634426 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b858251-2b59-440f-8bba-705df22c7ace admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:48.685260 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9b858251-2b59-440f-8bba-705df22c7ace admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c85953d8e7394fb0a6f425ce51420325'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:48.685915 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9b858251-2b59-440f-8bba-705df22c7ace admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c85953d8e7394fb0a6f425ce51420325'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:48.693835 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9b858251-2b59-440f-8bba-705df22c7ace admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c85953d8e7394fb0a6f425ce51420325'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:48.706076 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9b858251-2b59-440f-8bba-705df22c7ace admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c85953d8e7394fb0a6f425ce51420325'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:48.713469 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9b858251-2b59-440f-8bba-705df22c7ace admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c85953d8e7394fb0a6f425ce51420325'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:48.719534 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 841/1699] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:05:48 2022] DELETE /identity/v3/users/c85953d8e7394fb0a6f425ce51420325 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:48.890437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68867228-8a12-4b65-8558-2c5fb9a90451 service placement] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:48.901096 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68867228-8a12-4b65-8558-2c5fb9a90451 service placement] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:48.958078 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e291a-74d5-48b2-88ff-233f503e796c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:48.958703 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e291a-74d5-48b2-88ff-233f503e796c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:48.958979 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e291a-74d5-48b2-88ff-233f503e796c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:48.959243 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e291a-74d5-48b2-88ff-233f503e796c admin admin] PATH_INFO: `/v3/projects/72e1785381dd4bdaa2803fa7ef711dcb` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:48.960372 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e291a-74d5-48b2-88ff-233f503e796c admin admin] RBAC: Authorizing `identity:delete_project(project_id=72e1785381dd4bdaa2803fa7ef711dcb)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:48.962310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e291a-74d5-48b2-88ff-233f503e796c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:49.001467 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:49.009093 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2d9e291a-74d5-48b2-88ff-233f503e796c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72e1785381dd4bdaa2803fa7ef711dcb', 'request_id': 'req-2d9e291a-74d5-48b2-88ff-233f503e796c'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:49.014515 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2d9e291a-74d5-48b2-88ff-233f503e796c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72e1785381dd4bdaa2803fa7ef711dcb', 'request_id': 'req-2d9e291a-74d5-48b2-88ff-233f503e796c'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:49.023292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2d9e291a-74d5-48b2-88ff-233f503e796c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72e1785381dd4bdaa2803fa7ef711dcb', 'request_id': 'req-2d9e291a-74d5-48b2-88ff-233f503e796c'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:49.038881 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2d9e291a-74d5-48b2-88ff-233f503e796c admin admin] The token cache is being invalidate because project 72e1785381dd4bdaa2803fa7ef711dcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:05:49.039177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2d9e291a-74d5-48b2-88ff-233f503e796c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:49.039878 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2d9e291a-74d5-48b2-88ff-233f503e796c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:05:49.041837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 859/1700] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:05:48 2022] DELETE /identity/v3/projects/72e1785381dd4bdaa2803fa7ef711dcb => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:49.047058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea378a-c4fd-4efd-99f6-24f81b9e34ee None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:49.047363 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea378a-c4fd-4efd-99f6-24f81b9e34ee None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:49.047657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea378a-c4fd-4efd-99f6-24f81b9e34ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:49.068458 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-0eea378a-c4fd-4efd-99f6-24f81b9e34ee None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:49.069757 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0eea378a-c4fd-4efd-99f6-24f81b9e34ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:49.074250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:49.133900 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 842/1701] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:49.141763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f5d42-9d56-448c-b610-742252370dd0 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:49.142056 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f5d42-9d56-448c-b610-742252370dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:49.142680 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f5d42-9d56-448c-b610-742252370dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:49.165045 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-047f5d42-9d56-448c-b610-742252370dd0 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:49.166263 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-047f5d42-9d56-448c-b610-742252370dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:49.174757 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 860/1702] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:49.179044 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea378a-c4fd-4efd-99f6-24f81b9e34ee None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:49.188552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea378a-c4fd-4efd-99f6-24f81b9e34ee None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:49.193903 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506a471-e327-4368-bb7a-467e61d8f339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:49.194510 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506a471-e327-4368-bb7a-467e61d8f339 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:49.194791 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506a471-e327-4368-bb7a-467e61d8f339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:49.195065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506a471-e327-4368-bb7a-467e61d8f339 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:49.195709 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506a471-e327-4368-bb7a-467e61d8f339 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:05:49.196011 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506a471-e327-4368-bb7a-467e61d8f339 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:49.197646 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506a471-e327-4368-bb7a-467e61d8f339 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:49.209149 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 843/1703] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 06:05:49.213439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f5d42-9d56-448c-b610-742252370dd0 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:49.223205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f5d42-9d56-448c-b610-742252370dd0 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:49.229155 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c6a21-6005-4735-ab65-672a3d35498c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:49.229758 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c6a21-6005-4735-ab65-672a3d35498c admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:49.230038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c6a21-6005-4735-ab65-672a3d35498c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:49.230314 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c6a21-6005-4735-ab65-672a3d35498c admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:49.230941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c6a21-6005-4735-ab65-672a3d35498c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:49.232861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c6a21-6005-4735-ab65-672a3d35498c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:49.261888 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 861/1704] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:49 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:49.266034 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506a471-e327-4368-bb7a-467e61d8f339 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:49.276562 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506a471-e327-4368-bb7a-467e61d8f339 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:49.340612 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5847389-0111-4010-8e91-e2faff135f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:49.341323 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5847389-0111-4010-8e91-e2faff135f1c admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:49.341599 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5847389-0111-4010-8e91-e2faff135f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:49.341868 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5847389-0111-4010-8e91-e2faff135f1c admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:49.342504 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5847389-0111-4010-8e91-e2faff135f1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:49.344403 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5847389-0111-4010-8e91-e2faff135f1c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:49.370252 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 844/1705] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:49 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:05:49.374622 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c6a21-6005-4735-ab65-672a3d35498c admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:49.387187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c6a21-6005-4735-ab65-672a3d35498c admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:49.391805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a7d7c-8f4a-4140-b06c-9737de36b271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:49.393425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a7d7c-8f4a-4140-b06c-9737de36b271 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:49.393425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a7d7c-8f4a-4140-b06c-9737de36b271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:49.393649 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a7d7c-8f4a-4140-b06c-9737de36b271 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:49.394188 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a7d7c-8f4a-4140-b06c-9737de36b271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:49.395781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a7d7c-8f4a-4140-b06c-9737de36b271 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:49.405505 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 862/1706] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:49.409861 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5847389-0111-4010-8e91-e2faff135f1c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:49.419624 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5847389-0111-4010-8e91-e2faff135f1c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:49.424641 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fb3bb-69f3-4edf-8f33-c5f3f8af0944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:49.425265 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fb3bb-69f3-4edf-8f33-c5f3f8af0944 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:49.425543 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fb3bb-69f3-4edf-8f33-c5f3f8af0944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:49.425817 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fb3bb-69f3-4edf-8f33-c5f3f8af0944 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:49.426481 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7fb3bb-69f3-4edf-8f33-c5f3f8af0944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:49.428051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7fb3bb-69f3-4edf-8f33-c5f3f8af0944 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:49.438357 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 845/1707] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:49.442954 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a7d7c-8f4a-4140-b06c-9737de36b271 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:49.453219 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a7d7c-8f4a-4140-b06c-9737de36b271 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:49.459049 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51605816-0a74-4915-ad16-82a1e582162f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:49.459703 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51605816-0a74-4915-ad16-82a1e582162f admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:49.460037 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51605816-0a74-4915-ad16-82a1e582162f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:49.460233 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51605816-0a74-4915-ad16-82a1e582162f admin admin] PATH_INFO: `/v3/projects/8694b38528484b929b0bc17d529fdd27/users/0435070926d54818ba270c6279b69c2e/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:49.474790 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51605816-0a74-4915-ad16-82a1e582162f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8694b38528484b929b0bc17d529fdd27, user_id=0435070926d54818ba270c6279b69c2e, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:49.477485 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51605816-0a74-4915-ad16-82a1e582162f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:49.491436 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 863/1708] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:49 2022] PUT /identity/v3/projects/8694b38528484b929b0bc17d529fdd27/users/0435070926d54818ba270c6279b69c2e/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:49.503335 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe7afb-491f-4e72-858a-7a7080efdd6a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:49.503844 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe7afb-491f-4e72-858a-7a7080efdd6a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:49.503934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe7afb-491f-4e72-858a-7a7080efdd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:49.533839 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-1afe7afb-491f-4e72-858a-7a7080efdd6a None None] MFA Rules not processed for user `0435070926d54818ba270c6279b69c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:49.534884 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1afe7afb-491f-4e72-858a-7a7080efdd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:49.608900 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 846/1709] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:49.613803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51605816-0a74-4915-ad16-82a1e582162f admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:49.680032 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51605816-0a74-4915-ad16-82a1e582162f admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:49.684681 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4dad9-cf6c-40ec-ac9b-df3cfd4b71d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:49.685307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4dad9-cf6c-40ec-ac9b-df3cfd4b71d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:49.685586 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4dad9-cf6c-40ec-ac9b-df3cfd4b71d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:49.685857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4dad9-cf6c-40ec-ac9b-df3cfd4b71d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:49.686486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a4dad9-cf6c-40ec-ac9b-df3cfd4b71d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:49.688377 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a4dad9-cf6c-40ec-ac9b-df3cfd4b71d4 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:49.715721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 864/1710] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:05:49 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:05:49.719942 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe7afb-491f-4e72-858a-7a7080efdd6a None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:49.727186 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:49.731713 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe7afb-491f-4e72-858a-7a7080efdd6a None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:49.788284 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e73cf1-d7ab-4007-a70b-649ab418e887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:49.788840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e73cf1-d7ab-4007-a70b-649ab418e887 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:49.789124 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e73cf1-d7ab-4007-a70b-649ab418e887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:49.789399 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e73cf1-d7ab-4007-a70b-649ab418e887 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:49.790035 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e73cf1-d7ab-4007-a70b-649ab418e887 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:49.791954 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e73cf1-d7ab-4007-a70b-649ab418e887 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:49.818803 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 847/1711] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:05:49 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:05:49.819960 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4dad9-cf6c-40ec-ac9b-df3cfd4b71d4 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:49.830127 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4dad9-cf6c-40ec-ac9b-df3cfd4b71d4 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:49.835090 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221f219-34ae-4b2f-927a-9911836ff332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:49.835691 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221f219-34ae-4b2f-927a-9911836ff332 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:49.835968 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221f219-34ae-4b2f-927a-9911836ff332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:49.836239 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221f219-34ae-4b2f-927a-9911836ff332 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:49.836905 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a221f219-34ae-4b2f-927a-9911836ff332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:49.838471 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a221f219-34ae-4b2f-927a-9911836ff332 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:49.850565 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 865/1712] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:49.854976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e73cf1-d7ab-4007-a70b-649ab418e887 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:49.865863 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e73cf1-d7ab-4007-a70b-649ab418e887 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:49.870488 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf9b01-7675-4764-8163-89895e63a1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:49.871149 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf9b01-7675-4764-8163-89895e63a1ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:49.871427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf9b01-7675-4764-8163-89895e63a1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:49.871701 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf9b01-7675-4764-8163-89895e63a1ac admin admin] PATH_INFO: `/v3/projects/f8fbfd76f118456e8d40831b1ddb9aed/users/7296da353ff143ac9172097c234756ed/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:49.888559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf9b01-7675-4764-8163-89895e63a1ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8fbfd76f118456e8d40831b1ddb9aed, user_id=7296da353ff143ac9172097c234756ed, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:49.891440 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf9b01-7675-4764-8163-89895e63a1ac admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:49.905856 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 848/1713] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:49 2022] PUT /identity/v3/projects/f8fbfd76f118456e8d40831b1ddb9aed/users/7296da353ff143ac9172097c234756ed/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:49.910464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221f219-34ae-4b2f-927a-9911836ff332 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:49.920395 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221f219-34ae-4b2f-927a-9911836ff332 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:49.965445 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:05:49.990341 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27467b-f9f2-4dbb-af60-239943732097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:49.990750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27467b-f9f2-4dbb-af60-239943732097 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:49.991030 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27467b-f9f2-4dbb-af60-239943732097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:49.991303 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27467b-f9f2-4dbb-af60-239943732097 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:49.991946 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27467b-f9f2-4dbb-af60-239943732097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:49.993554 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27467b-f9f2-4dbb-af60-239943732097 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:50.007415 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 866/1714] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:05:50.008864 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf9b01-7675-4764-8163-89895e63a1ac admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:50.019845 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf9b01-7675-4764-8163-89895e63a1ac admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:50.024955 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f11ce-e85f-4415-8964-a8cdc2ea16aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:50.025628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f11ce-e85f-4415-8964-a8cdc2ea16aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:50.025903 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f11ce-e85f-4415-8964-a8cdc2ea16aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:50.026173 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f11ce-e85f-4415-8964-a8cdc2ea16aa admin admin] PATH_INFO: `/v3/projects/f8fbfd76f118456e8d40831b1ddb9aed/users/7296da353ff143ac9172097c234756ed/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:50.028081 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f11ce-e85f-4415-8964-a8cdc2ea16aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8fbfd76f118456e8d40831b1ddb9aed, user_id=7296da353ff143ac9172097c234756ed, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:50.030799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f11ce-e85f-4415-8964-a8cdc2ea16aa admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:50.042567 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 849/1715] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:05:50 2022] PUT /identity/v3/projects/f8fbfd76f118456e8d40831b1ddb9aed/users/7296da353ff143ac9172097c234756ed/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:05:50.057016 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94087e46-5d4d-4dc5-858c-4c4944135d34 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:50.057315 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94087e46-5d4d-4dc5-858c-4c4944135d34 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:50.057566 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94087e46-5d4d-4dc5-858c-4c4944135d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:50.089321 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-94087e46-5d4d-4dc5-858c-4c4944135d34 None None] MFA Rules not processed for user `7296da353ff143ac9172097c234756ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:05:50.090665 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-94087e46-5d4d-4dc5-858c-4c4944135d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:50.176497 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 867/1716] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:05:50 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:05:50.188623 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f11ce-e85f-4415-8964-a8cdc2ea16aa admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:50.190897 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-856f11ce-e85f-4415-8964-a8cdc2ea16aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:05:50.248450 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f11ce-e85f-4415-8964-a8cdc2ea16aa admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:50.253080 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820df6e7-4d00-4069-91cc-3ea2f9899a35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:50.253682 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820df6e7-4d00-4069-91cc-3ea2f9899a35 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:50.253958 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820df6e7-4d00-4069-91cc-3ea2f9899a35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:50.254225 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820df6e7-4d00-4069-91cc-3ea2f9899a35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:50.263642 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820df6e7-4d00-4069-91cc-3ea2f9899a35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:50.265120 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820df6e7-4d00-4069-91cc-3ea2f9899a35 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:50.275105 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 850/1717] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:50 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:05:50.351325 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94087e46-5d4d-4dc5-858c-4c4944135d34 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:05:50.360890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94087e46-5d4d-4dc5-858c-4c4944135d34 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:05:50.365560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d585b-5e40-4155-ba7b-08e6d56616b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:05:50.366169 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d585b-5e40-4155-ba7b-08e6d56616b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:05:50.366526 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d585b-5e40-4155-ba7b-08e6d56616b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:05:50.366694 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d585b-5e40-4155-ba7b-08e6d56616b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:05:50.425467 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d585b-5e40-4155-ba7b-08e6d56616b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:05:50.426944 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d585b-5e40-4155-ba7b-08e6d56616b0 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:05:50.435669 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 868/1718] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:05:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:06:00.270954 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=68,keystone:INSERT=5 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:00.431586 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=73 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:04.966869 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820df6e7-4d00-4069-91cc-3ea2f9899a35 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:05.036632 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820df6e7-4d00-4069-91cc-3ea2f9899a35 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:05.041537 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cac482-66e3-42be-8642-0ee2f0016e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:05.042174 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cac482-66e3-42be-8642-0ee2f0016e5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:05.042450 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cac482-66e3-42be-8642-0ee2f0016e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:05.042727 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cac482-66e3-42be-8642-0ee2f0016e5c admin admin] PATH_INFO: `/v3/users/0435070926d54818ba270c6279b69c2e` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:05.043843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cac482-66e3-42be-8642-0ee2f0016e5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0435070926d54818ba270c6279b69c2e)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:05.045785 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cac482-66e3-42be-8642-0ee2f0016e5c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:05.090857 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c8cac482-66e3-42be-8642-0ee2f0016e5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0435070926d54818ba270c6279b69c2e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:05.091500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c8cac482-66e3-42be-8642-0ee2f0016e5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0435070926d54818ba270c6279b69c2e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:05.099651 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c8cac482-66e3-42be-8642-0ee2f0016e5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0435070926d54818ba270c6279b69c2e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:05.111922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c8cac482-66e3-42be-8642-0ee2f0016e5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0435070926d54818ba270c6279b69c2e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:05.118850 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c8cac482-66e3-42be-8642-0ee2f0016e5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0435070926d54818ba270c6279b69c2e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:05.124765 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 851/1719] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:06:04 2022] DELETE /identity/v3/users/0435070926d54818ba270c6279b69c2e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:05.129227 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8d419-bb3e-4a64-af39-c0bed0202157 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:05.130185 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8d419-bb3e-4a64-af39-c0bed0202157 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:05.130436 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8d419-bb3e-4a64-af39-c0bed0202157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:05.150476 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-3ec8d419-bb3e-4a64-af39-c0bed0202157 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:05.151856 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3ec8d419-bb3e-4a64-af39-c0bed0202157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:05.209719 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 869/1720] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:06:05.219240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cac482-66e3-42be-8642-0ee2f0016e5c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:05.220671 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c8cac482-66e3-42be-8642-0ee2f0016e5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:05.280549 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cac482-66e3-42be-8642-0ee2f0016e5c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:05.284263 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27356b64-b3ee-40e3-83b6-dcc4a7a4db03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:05.284900 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27356b64-b3ee-40e3-83b6-dcc4a7a4db03 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:05.285178 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27356b64-b3ee-40e3-83b6-dcc4a7a4db03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:05.285448 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27356b64-b3ee-40e3-83b6-dcc4a7a4db03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:05.296771 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27356b64-b3ee-40e3-83b6-dcc4a7a4db03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:05.298226 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27356b64-b3ee-40e3-83b6-dcc4a7a4db03 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:05.309814 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 852/1721] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:06:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:06:05.575178 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8d419-bb3e-4a64-af39-c0bed0202157 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:05.581243 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3ec8d419-bb3e-4a64-af39-c0bed0202157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:05.595827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8d419-bb3e-4a64-af39-c0bed0202157 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:05.600978 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05608776-779f-422b-b0a0-0031ca4fd044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:05.601627 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05608776-779f-422b-b0a0-0031ca4fd044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:05.601898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05608776-779f-422b-b0a0-0031ca4fd044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:05.602168 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05608776-779f-422b-b0a0-0031ca4fd044 admin admin] PATH_INFO: `/v3/projects/8694b38528484b929b0bc17d529fdd27` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:05.609449 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05608776-779f-422b-b0a0-0031ca4fd044 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8694b38528484b929b0bc17d529fdd27)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:05.611352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05608776-779f-422b-b0a0-0031ca4fd044 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:05.658388 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-05608776-779f-422b-b0a0-0031ca4fd044 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8694b38528484b929b0bc17d529fdd27', 'request_id': 'req-05608776-779f-422b-b0a0-0031ca4fd044'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:05.664190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-05608776-779f-422b-b0a0-0031ca4fd044 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8694b38528484b929b0bc17d529fdd27', 'request_id': 'req-05608776-779f-422b-b0a0-0031ca4fd044'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:05.670466 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-05608776-779f-422b-b0a0-0031ca4fd044 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8694b38528484b929b0bc17d529fdd27', 'request_id': 'req-05608776-779f-422b-b0a0-0031ca4fd044'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:05.682166 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-05608776-779f-422b-b0a0-0031ca4fd044 admin admin] The token cache is being invalidate because project 8694b38528484b929b0bc17d529fdd27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:06:05.682452 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-05608776-779f-422b-b0a0-0031ca4fd044 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:05.683063 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-05608776-779f-422b-b0a0-0031ca4fd044 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:05.684522 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 870/1722] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:06:05 2022] DELETE /identity/v3/projects/8694b38528484b929b0bc17d529fdd27 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:05.688572 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27356b64-b3ee-40e3-83b6-dcc4a7a4db03 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:05.698718 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27356b64-b3ee-40e3-83b6-dcc4a7a4db03 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:05.717210 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:05.758163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45c7e6-eae2-4947-b5ab-928fe39b8a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:05.759054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45c7e6-eae2-4947-b5ab-928fe39b8a02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:05.759335 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45c7e6-eae2-4947-b5ab-928fe39b8a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:05.759611 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45c7e6-eae2-4947-b5ab-928fe39b8a02 admin admin] PATH_INFO: `/v3/users/7296da353ff143ac9172097c234756ed` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:05.760761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff45c7e6-eae2-4947-b5ab-928fe39b8a02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7296da353ff143ac9172097c234756ed)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:05.762690 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff45c7e6-eae2-4947-b5ab-928fe39b8a02 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:05.810180 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ff45c7e6-eae2-4947-b5ab-928fe39b8a02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7296da353ff143ac9172097c234756ed'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:05.810969 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ff45c7e6-eae2-4947-b5ab-928fe39b8a02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7296da353ff143ac9172097c234756ed'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:05.822779 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ff45c7e6-eae2-4947-b5ab-928fe39b8a02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7296da353ff143ac9172097c234756ed'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:05.836560 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ff45c7e6-eae2-4947-b5ab-928fe39b8a02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7296da353ff143ac9172097c234756ed'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:05.843534 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ff45c7e6-eae2-4947-b5ab-928fe39b8a02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7296da353ff143ac9172097c234756ed'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:05.852588 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 853/1723] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:06:05 2022] DELETE /identity/v3/users/7296da353ff143ac9172097c234756ed => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:06.134174 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05608776-779f-422b-b0a0-0031ca4fd044 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:06.135648 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-05608776-779f-422b-b0a0-0031ca4fd044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:06.150211 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05608776-779f-422b-b0a0-0031ca4fd044 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:06.222837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37e7de-5264-41aa-b421-3d79d88480f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:06.224510 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37e7de-5264-41aa-b421-3d79d88480f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:06.224874 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37e7de-5264-41aa-b421-3d79d88480f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:06.225148 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37e7de-5264-41aa-b421-3d79d88480f0 admin admin] PATH_INFO: `/v3/projects/f8fbfd76f118456e8d40831b1ddb9aed` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:06.226298 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37e7de-5264-41aa-b421-3d79d88480f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8fbfd76f118456e8d40831b1ddb9aed)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:06.228238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37e7de-5264-41aa-b421-3d79d88480f0 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:06.262740 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:06.281043 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-dd37e7de-5264-41aa-b421-3d79d88480f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8fbfd76f118456e8d40831b1ddb9aed', 'request_id': 'req-dd37e7de-5264-41aa-b421-3d79d88480f0'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:06.287321 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-dd37e7de-5264-41aa-b421-3d79d88480f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8fbfd76f118456e8d40831b1ddb9aed', 'request_id': 'req-dd37e7de-5264-41aa-b421-3d79d88480f0'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:06.293597 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-dd37e7de-5264-41aa-b421-3d79d88480f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8fbfd76f118456e8d40831b1ddb9aed', 'request_id': 'req-dd37e7de-5264-41aa-b421-3d79d88480f0'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:06.307383 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-dd37e7de-5264-41aa-b421-3d79d88480f0 admin admin] The token cache is being invalidate because project f8fbfd76f118456e8d40831b1ddb9aed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:06:06.307728 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-dd37e7de-5264-41aa-b421-3d79d88480f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:06.308583 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-dd37e7de-5264-41aa-b421-3d79d88480f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:06.309855 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 871/1724] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:06:06 2022] DELETE /identity/v3/projects/f8fbfd76f118456e8d40831b1ddb9aed => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:06.315086 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2618d-81cb-4d17-a670-08524faa30db None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:06.315557 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2618d-81cb-4d17-a670-08524faa30db None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:06.315824 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2618d-81cb-4d17-a670-08524faa30db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:06.338613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-a1f2618d-81cb-4d17-a670-08524faa30db None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:06.339537 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a1f2618d-81cb-4d17-a670-08524faa30db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:06.403228 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 854/1725] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:06:06.411410 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0705d5-5268-4958-b561-ca213908b786 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:06.411714 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0705d5-5268-4958-b561-ca213908b786 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:06.411992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0705d5-5268-4958-b561-ca213908b786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:06.432076 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-6e0705d5-5268-4958-b561-ca213908b786 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:06.433338 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6e0705d5-5268-4958-b561-ca213908b786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:06.441396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 872/1726] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:06:06.446827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2618d-81cb-4d17-a670-08524faa30db None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:06.457102 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2618d-81cb-4d17-a670-08524faa30db None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:06.464688 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b37ed-5b6b-4f56-9caf-101b2b426285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:06.465686 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b37ed-5b6b-4f56-9caf-101b2b426285 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:06.466108 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b37ed-5b6b-4f56-9caf-101b2b426285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:06.466388 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b37ed-5b6b-4f56-9caf-101b2b426285 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:06.467045 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b37ed-5b6b-4f56-9caf-101b2b426285 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:06:06.467359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b37ed-5b6b-4f56-9caf-101b2b426285 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:06.469044 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b37ed-5b6b-4f56-9caf-101b2b426285 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:06.481964 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 855/1727] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:06:06.486527 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0705d5-5268-4958-b561-ca213908b786 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:06.497299 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0705d5-5268-4958-b561-ca213908b786 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:06.503144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253ac19-9cc9-4fa9-920b-5a06cd790fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:06.503757 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253ac19-9cc9-4fa9-920b-5a06cd790fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:06.504037 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253ac19-9cc9-4fa9-920b-5a06cd790fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:06.504318 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253ac19-9cc9-4fa9-920b-5a06cd790fd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:06.505049 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4253ac19-9cc9-4fa9-920b-5a06cd790fd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:06.506974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4253ac19-9cc9-4fa9-920b-5a06cd790fd6 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:06.539244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 873/1728] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:06:06 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:06:06.543513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b37ed-5b6b-4f56-9caf-101b2b426285 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:06.554247 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b37ed-5b6b-4f56-9caf-101b2b426285 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:06.583162 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:06.622231 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072962c-ed3c-460d-91f7-a4619d0b0ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:06.622849 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072962c-ed3c-460d-91f7-a4619d0b0ead admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:06.623124 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072962c-ed3c-460d-91f7-a4619d0b0ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:06.623395 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072962c-ed3c-460d-91f7-a4619d0b0ead admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:06.624053 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b072962c-ed3c-460d-91f7-a4619d0b0ead admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:06.625990 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b072962c-ed3c-460d-91f7-a4619d0b0ead admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:06.647512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 856/1729] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:06:06 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:06:06.651497 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253ac19-9cc9-4fa9-920b-5a06cd790fd6 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:06.662790 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253ac19-9cc9-4fa9-920b-5a06cd790fd6 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:06.667543 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0982da50-d90c-4bfe-a9b5-eec94736a82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:06.668136 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982da50-d90c-4bfe-a9b5-eec94736a82d admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:06.668413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982da50-d90c-4bfe-a9b5-eec94736a82d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:06.668686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982da50-d90c-4bfe-a9b5-eec94736a82d admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:06.669506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0982da50-d90c-4bfe-a9b5-eec94736a82d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:06.671097 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0982da50-d90c-4bfe-a9b5-eec94736a82d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:06.682101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 874/1730] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:06:06.686015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072962c-ed3c-460d-91f7-a4619d0b0ead admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:06.696274 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072962c-ed3c-460d-91f7-a4619d0b0ead admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:06.700899 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447812e-5789-404a-bed4-cf77a322abe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:06.701491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447812e-5789-404a-bed4-cf77a322abe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:06.701768 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447812e-5789-404a-bed4-cf77a322abe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:06.702033 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447812e-5789-404a-bed4-cf77a322abe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:06.702688 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a447812e-5789-404a-bed4-cf77a322abe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:06.704252 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a447812e-5789-404a-bed4-cf77a322abe1 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:06.715096 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 857/1731] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:06:06.719086 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0982da50-d90c-4bfe-a9b5-eec94736a82d admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:06.729618 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0982da50-d90c-4bfe-a9b5-eec94736a82d admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:06.734262 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca91b31-0908-438d-961b-0a248bd2a395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:06.734903 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca91b31-0908-438d-961b-0a248bd2a395 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:06.735179 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca91b31-0908-438d-961b-0a248bd2a395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:06.735451 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca91b31-0908-438d-961b-0a248bd2a395 admin admin] PATH_INFO: `/v3/projects/26904818f5f74f6da3110b3fc786390c/users/d272e37a6e244c5092f0406e4a854368/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:06.750078 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca91b31-0908-438d-961b-0a248bd2a395 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26904818f5f74f6da3110b3fc786390c, user_id=d272e37a6e244c5092f0406e4a854368, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:06.752807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca91b31-0908-438d-961b-0a248bd2a395 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:06.766012 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 875/1732] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:06:06 2022] PUT /identity/v3/projects/26904818f5f74f6da3110b3fc786390c/users/d272e37a6e244c5092f0406e4a854368/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:06.779123 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47ecee-c2a0-4fdb-b703-dcdfc466b22a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:06.779624 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47ecee-c2a0-4fdb-b703-dcdfc466b22a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:06.780140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47ecee-c2a0-4fdb-b703-dcdfc466b22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:06.809850 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-fe47ecee-c2a0-4fdb-b703-dcdfc466b22a None None] MFA Rules not processed for user `d272e37a6e244c5092f0406e4a854368`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:06.811089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fe47ecee-c2a0-4fdb-b703-dcdfc466b22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:06.873154 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 858/1733] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:06:06.877697 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca91b31-0908-438d-961b-0a248bd2a395 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:06.938738 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca91b31-0908-438d-961b-0a248bd2a395 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:06.943640 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43959a0-e918-4324-8555-23d7710fc650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:06.944234 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43959a0-e918-4324-8555-23d7710fc650 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:06.944511 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43959a0-e918-4324-8555-23d7710fc650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:06.944815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43959a0-e918-4324-8555-23d7710fc650 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:06.945453 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43959a0-e918-4324-8555-23d7710fc650 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:06.947323 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43959a0-e918-4324-8555-23d7710fc650 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:06.976243 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 876/1734] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:06:06 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:06:06.980490 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47ecee-c2a0-4fdb-b703-dcdfc466b22a None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:06.991250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47ecee-c2a0-4fdb-b703-dcdfc466b22a None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:07.050393 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff1c54-1d33-48f7-9231-8d6f0bbabeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:07.051012 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff1c54-1d33-48f7-9231-8d6f0bbabeb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:07.051298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff1c54-1d33-48f7-9231-8d6f0bbabeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:07.051683 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff1c54-1d33-48f7-9231-8d6f0bbabeb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:07.052226 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff1c54-1d33-48f7-9231-8d6f0bbabeb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:07.054165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff1c54-1d33-48f7-9231-8d6f0bbabeb8 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:07.076442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 859/1735] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:06:06 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:06:07.080256 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43959a0-e918-4324-8555-23d7710fc650 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:07.091632 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43959a0-e918-4324-8555-23d7710fc650 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:07.096327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6882dd-d688-4dd3-b73e-07ec3270253c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:07.096963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6882dd-d688-4dd3-b73e-07ec3270253c admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:07.097245 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6882dd-d688-4dd3-b73e-07ec3270253c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:07.097520 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6882dd-d688-4dd3-b73e-07ec3270253c admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:07.098187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6882dd-d688-4dd3-b73e-07ec3270253c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:07.100033 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6882dd-d688-4dd3-b73e-07ec3270253c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:07.113816 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 877/1736] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:06:07.115187 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff1c54-1d33-48f7-9231-8d6f0bbabeb8 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:07.125732 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff1c54-1d33-48f7-9231-8d6f0bbabeb8 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:07.131360 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88841f36-b951-4736-ac35-71db49c4db2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:07.132171 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88841f36-b951-4736-ac35-71db49c4db2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:07.132453 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88841f36-b951-4736-ac35-71db49c4db2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:07.133039 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88841f36-b951-4736-ac35-71db49c4db2c admin admin] PATH_INFO: `/v3/projects/bb487342df344999befff30eaf63483c/users/72d680cfbe6a49ce8eb22aa053879041/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:07.149155 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88841f36-b951-4736-ac35-71db49c4db2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb487342df344999befff30eaf63483c, user_id=72d680cfbe6a49ce8eb22aa053879041, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:07.151838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88841f36-b951-4736-ac35-71db49c4db2c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:07.168517 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 860/1737] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:06:07 2022] PUT /identity/v3/projects/bb487342df344999befff30eaf63483c/users/72d680cfbe6a49ce8eb22aa053879041/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:07.172653 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6882dd-d688-4dd3-b73e-07ec3270253c admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:07.185451 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6882dd-d688-4dd3-b73e-07ec3270253c admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:07.210271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:07.266894 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12e6cb-f230-4e45-b87e-c8d243bf07bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:07.267513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12e6cb-f230-4e45-b87e-c8d243bf07bd admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:07.267787 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12e6cb-f230-4e45-b87e-c8d243bf07bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:07.268049 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12e6cb-f230-4e45-b87e-c8d243bf07bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:07.268879 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12e6cb-f230-4e45-b87e-c8d243bf07bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:07.270753 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12e6cb-f230-4e45-b87e-c8d243bf07bd admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:07.283960 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 878/1738] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:06:07.288253 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88841f36-b951-4736-ac35-71db49c4db2c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:07.293642 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:07.301324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88841f36-b951-4736-ac35-71db49c4db2c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:07.320904 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4fd928-d5bf-4694-948d-7d34496bcc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:07.321597 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fd928-d5bf-4694-948d-7d34496bcc07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:07.321912 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fd928-d5bf-4694-948d-7d34496bcc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:07.322221 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fd928-d5bf-4694-948d-7d34496bcc07 admin admin] PATH_INFO: `/v3/projects/bb487342df344999befff30eaf63483c/users/72d680cfbe6a49ce8eb22aa053879041/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:07.324396 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4fd928-d5bf-4694-948d-7d34496bcc07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb487342df344999befff30eaf63483c, user_id=72d680cfbe6a49ce8eb22aa053879041, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:07.327287 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4fd928-d5bf-4694-948d-7d34496bcc07 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:07.338222 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 861/1739] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:06:07 2022] PUT /identity/v3/projects/bb487342df344999befff30eaf63483c/users/72d680cfbe6a49ce8eb22aa053879041/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:07.349030 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a03f9-cb97-417a-9ff8-f3d63358c201 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:07.349312 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a03f9-cb97-417a-9ff8-f3d63358c201 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:07.349623 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a03f9-cb97-417a-9ff8-f3d63358c201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:07.381009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-089a03f9-cb97-417a-9ff8-f3d63358c201 None None] MFA Rules not processed for user `72d680cfbe6a49ce8eb22aa053879041`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:07.382526 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-089a03f9-cb97-417a-9ff8-f3d63358c201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:07.455139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 879/1740] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:06:07.467176 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4fd928-d5bf-4694-948d-7d34496bcc07 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:07.468920 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3a4fd928-d5bf-4694-948d-7d34496bcc07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:07.531459 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4fd928-d5bf-4694-948d-7d34496bcc07 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:07.537255 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462868e4-1654-4932-bf84-1d70bd69a663 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:07.537863 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462868e4-1654-4932-bf84-1d70bd69a663 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:07.538143 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462868e4-1654-4932-bf84-1d70bd69a663 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:07.538414 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462868e4-1654-4932-bf84-1d70bd69a663 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:07.548838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462868e4-1654-4932-bf84-1d70bd69a663 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:07.550715 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462868e4-1654-4932-bf84-1d70bd69a663 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:07.561882 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 862/1741] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:06:07 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:06:07.631816 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a03f9-cb97-417a-9ff8-f3d63358c201 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:07.693706 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a03f9-cb97-417a-9ff8-f3d63358c201 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:07.698359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550261e-2fde-485c-ad23-e2a45011b2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:07.698989 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550261e-2fde-485c-ad23-e2a45011b2f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:07.699260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550261e-2fde-485c-ad23-e2a45011b2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:07.699524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550261e-2fde-485c-ad23-e2a45011b2f1 admin admin] PATH_INFO: `/v3/users/d272e37a6e244c5092f0406e4a854368` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:07.700638 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550261e-2fde-485c-ad23-e2a45011b2f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d272e37a6e244c5092f0406e4a854368)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:07.702567 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550261e-2fde-485c-ad23-e2a45011b2f1 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:07.747670 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-e550261e-2fde-485c-ad23-e2a45011b2f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd272e37a6e244c5092f0406e4a854368'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:07.748308 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-e550261e-2fde-485c-ad23-e2a45011b2f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd272e37a6e244c5092f0406e4a854368'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:07.749951 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:07.763114 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-e550261e-2fde-485c-ad23-e2a45011b2f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd272e37a6e244c5092f0406e4a854368'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:07.782225 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-e550261e-2fde-485c-ad23-e2a45011b2f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd272e37a6e244c5092f0406e4a854368'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:07.782885 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-e550261e-2fde-485c-ad23-e2a45011b2f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd272e37a6e244c5092f0406e4a854368'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:07.795055 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 880/1742] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:06:07 2022] DELETE /identity/v3/users/d272e37a6e244c5092f0406e4a854368 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:07.799702 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce077c-51df-4a48-9208-8c58b5c789d7 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:07.799993 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce077c-51df-4a48-9208-8c58b5c789d7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:07.800269 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce077c-51df-4a48-9208-8c58b5c789d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:07.819983 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-d0ce077c-51df-4a48-9208-8c58b5c789d7 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:07.821226 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d0ce077c-51df-4a48-9208-8c58b5c789d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:07.879478 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 863/1743] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:06:07.889575 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550261e-2fde-485c-ad23-e2a45011b2f1 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:07.890962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e550261e-2fde-485c-ad23-e2a45011b2f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:07.948916 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550261e-2fde-485c-ad23-e2a45011b2f1 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:07.952545 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff303c-35a2-4ca7-994c-0cb054148f4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:07.953173 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff303c-35a2-4ca7-994c-0cb054148f4e service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:07.953451 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff303c-35a2-4ca7-994c-0cb054148f4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:07.953720 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff303c-35a2-4ca7-994c-0cb054148f4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:07.963263 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff303c-35a2-4ca7-994c-0cb054148f4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:07.964785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff303c-35a2-4ca7-994c-0cb054148f4e service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:07.976614 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 881/1744] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:06:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:06:08.240253 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce077c-51df-4a48-9208-8c58b5c789d7 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:08.241830 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d0ce077c-51df-4a48-9208-8c58b5c789d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:08.254586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce077c-51df-4a48-9208-8c58b5c789d7 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:08.277986 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83184da-58d9-46e4-a373-e94cb793c3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:08.278706 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83184da-58d9-46e4-a373-e94cb793c3a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:08.279050 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83184da-58d9-46e4-a373-e94cb793c3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:08.279403 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83184da-58d9-46e4-a373-e94cb793c3a1 admin admin] PATH_INFO: `/v3/projects/26904818f5f74f6da3110b3fc786390c` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:08.280532 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83184da-58d9-46e4-a373-e94cb793c3a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26904818f5f74f6da3110b3fc786390c)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:08.282451 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83184da-58d9-46e4-a373-e94cb793c3a1 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:08.337728 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a83184da-58d9-46e4-a373-e94cb793c3a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26904818f5f74f6da3110b3fc786390c', 'request_id': 'req-a83184da-58d9-46e4-a373-e94cb793c3a1'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:08.344273 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a83184da-58d9-46e4-a373-e94cb793c3a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26904818f5f74f6da3110b3fc786390c', 'request_id': 'req-a83184da-58d9-46e4-a373-e94cb793c3a1'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:08.353385 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a83184da-58d9-46e4-a373-e94cb793c3a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26904818f5f74f6da3110b3fc786390c', 'request_id': 'req-a83184da-58d9-46e4-a373-e94cb793c3a1'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:08.379160 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a83184da-58d9-46e4-a373-e94cb793c3a1 admin admin] The token cache is being invalidate because project 26904818f5f74f6da3110b3fc786390c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:06:08.379755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a83184da-58d9-46e4-a373-e94cb793c3a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:08.382721 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a83184da-58d9-46e4-a373-e94cb793c3a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:08.385735 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 864/1745] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:06:08 2022] DELETE /identity/v3/projects/26904818f5f74f6da3110b3fc786390c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:08.389845 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff303c-35a2-4ca7-994c-0cb054148f4e service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:08.401846 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff303c-35a2-4ca7-994c-0cb054148f4e service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:08.432963 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83184da-58d9-46e4-a373-e94cb793c3a1 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:08.434101 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a83184da-58d9-46e4-a373-e94cb793c3a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:08.456089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:08.499325 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5e2d0-5bca-40a0-8d88-9b02f5e2ec23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:08.500098 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5e2d0-5bca-40a0-8d88-9b02f5e2ec23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:08.500578 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5e2d0-5bca-40a0-8d88-9b02f5e2ec23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:08.501011 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5e2d0-5bca-40a0-8d88-9b02f5e2ec23 admin admin] PATH_INFO: `/v3/users/72d680cfbe6a49ce8eb22aa053879041` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:08.502374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a5e2d0-5bca-40a0-8d88-9b02f5e2ec23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72d680cfbe6a49ce8eb22aa053879041)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:08.504311 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a5e2d0-5bca-40a0-8d88-9b02f5e2ec23 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:08.511666 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83184da-58d9-46e4-a373-e94cb793c3a1 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:08.518425 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b181f0-f892-465d-a6c6-ff5684fe19c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:08.519078 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b181f0-f892-465d-a6c6-ff5684fe19c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:08.519357 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b181f0-f892-465d-a6c6-ff5684fe19c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:08.519627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b181f0-f892-465d-a6c6-ff5684fe19c0 admin admin] PATH_INFO: `/v3/users/e3bd53f6a4d949df8313ad093c0a5987` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:08.521932 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b181f0-f892-465d-a6c6-ff5684fe19c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3bd53f6a4d949df8313ad093c0a5987)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:08.524226 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b181f0-f892-465d-a6c6-ff5684fe19c0 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:08.569137 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a7a5e2d0-5bca-40a0-8d88-9b02f5e2ec23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72d680cfbe6a49ce8eb22aa053879041'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:08.570658 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a7a5e2d0-5bca-40a0-8d88-9b02f5e2ec23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72d680cfbe6a49ce8eb22aa053879041'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:08.581283 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-39b181f0-f892-465d-a6c6-ff5684fe19c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3bd53f6a4d949df8313ad093c0a5987'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:08.582179 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-39b181f0-f892-465d-a6c6-ff5684fe19c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3bd53f6a4d949df8313ad093c0a5987'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:08.586968 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a7a5e2d0-5bca-40a0-8d88-9b02f5e2ec23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72d680cfbe6a49ce8eb22aa053879041'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:08.598303 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-39b181f0-f892-465d-a6c6-ff5684fe19c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3bd53f6a4d949df8313ad093c0a5987'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:08.605633 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:08.607793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a7a5e2d0-5bca-40a0-8d88-9b02f5e2ec23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72d680cfbe6a49ce8eb22aa053879041'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:08.610844 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a7a5e2d0-5bca-40a0-8d88-9b02f5e2ec23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72d680cfbe6a49ce8eb22aa053879041'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:08.628721 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-39b181f0-f892-465d-a6c6-ff5684fe19c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3bd53f6a4d949df8313ad093c0a5987'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:08.628721 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-39b181f0-f892-465d-a6c6-ff5684fe19c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3bd53f6a4d949df8313ad093c0a5987'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:08.628952 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 882/1746] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:06:08 2022] DELETE /identity/v3/users/72d680cfbe6a49ce8eb22aa053879041 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:08.636738 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 865/1747] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:06:08 2022] DELETE /identity/v3/users/e3bd53f6a4d949df8313ad093c0a5987 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:06:08.641144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa9311-7e2a-4932-a8ba-c7ee8ed6305f None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:08.641443 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa9311-7e2a-4932-a8ba-c7ee8ed6305f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:08.641720 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa9311-7e2a-4932-a8ba-c7ee8ed6305f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:08.663673 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-dffa9311-7e2a-4932-a8ba-c7ee8ed6305f None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:08.664964 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-dffa9311-7e2a-4932-a8ba-c7ee8ed6305f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:08.729533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 883/1748] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:06:08.739464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b181f0-f892-465d-a6c6-ff5684fe19c0 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:08.741302 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-39b181f0-f892-465d-a6c6-ff5684fe19c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:08.813485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b181f0-f892-465d-a6c6-ff5684fe19c0 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:08.817759 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85793800-6506-4409-8d78-a70c18f7b1b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:08.818412 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85793800-6506-4409-8d78-a70c18f7b1b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:08.818747 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85793800-6506-4409-8d78-a70c18f7b1b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:08.819018 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85793800-6506-4409-8d78-a70c18f7b1b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:08.828119 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85793800-6506-4409-8d78-a70c18f7b1b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:08.829615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85793800-6506-4409-8d78-a70c18f7b1b0 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:08.839679 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 866/1749] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:06:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:06:09.023308 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa9311-7e2a-4932-a8ba-c7ee8ed6305f None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:09.024798 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-dffa9311-7e2a-4932-a8ba-c7ee8ed6305f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:09.040498 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa9311-7e2a-4932-a8ba-c7ee8ed6305f None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:09.045244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49732335-bc76-469f-a036-f18cbe7193d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:09.045951 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49732335-bc76-469f-a036-f18cbe7193d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:09.046243 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49732335-bc76-469f-a036-f18cbe7193d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:09.047669 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49732335-bc76-469f-a036-f18cbe7193d3 admin admin] PATH_INFO: `/v3/projects/bb487342df344999befff30eaf63483c` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:09.048628 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49732335-bc76-469f-a036-f18cbe7193d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb487342df344999befff30eaf63483c)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:09.050987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49732335-bc76-469f-a036-f18cbe7193d3 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:09.123062 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-49732335-bc76-469f-a036-f18cbe7193d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb487342df344999befff30eaf63483c', 'request_id': 'req-49732335-bc76-469f-a036-f18cbe7193d3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:09.129380 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-49732335-bc76-469f-a036-f18cbe7193d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb487342df344999befff30eaf63483c', 'request_id': 'req-49732335-bc76-469f-a036-f18cbe7193d3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:09.138196 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-49732335-bc76-469f-a036-f18cbe7193d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb487342df344999befff30eaf63483c', 'request_id': 'req-49732335-bc76-469f-a036-f18cbe7193d3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:09.157128 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85793800-6506-4409-8d78-a70c18f7b1b0 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:09.158481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-49732335-bc76-469f-a036-f18cbe7193d3 admin admin] The token cache is being invalidate because project bb487342df344999befff30eaf63483c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:06:09.158598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-85793800-6506-4409-8d78-a70c18f7b1b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:09.159281 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-49732335-bc76-469f-a036-f18cbe7193d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:09.160272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-49732335-bc76-469f-a036-f18cbe7193d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:09.163426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 884/1750] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:06:09 2022] DELETE /identity/v3/projects/bb487342df344999befff30eaf63483c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:09.168768 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1da7f5-ce20-4846-855d-9500d60bd779 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:09.169200 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1da7f5-ce20-4846-855d-9500d60bd779 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:09.169733 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1da7f5-ce20-4846-855d-9500d60bd779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:09.194848 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-cc1da7f5-ce20-4846-855d-9500d60bd779 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:09.196146 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-cc1da7f5-ce20-4846-855d-9500d60bd779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:09.206061 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85793800-6506-4409-8d78-a70c18f7b1b0 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:09.268741 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:09.289115 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 885/1751] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:06:09.293040 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da56577-94e2-40af-8d78-67eb74481df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:09.293688 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da56577-94e2-40af-8d78-67eb74481df6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:09.293962 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da56577-94e2-40af-8d78-67eb74481df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:09.294229 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da56577-94e2-40af-8d78-67eb74481df6 admin admin] PATH_INFO: `/v3/projects/7457566282144956bff50037d25737dc` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:09.296132 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da56577-94e2-40af-8d78-67eb74481df6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7457566282144956bff50037d25737dc)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:09.298414 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da56577-94e2-40af-8d78-67eb74481df6 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:09.305679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef18ec9-3d53-48a9-81a9-9b036336d428 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:09.305968 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef18ec9-3d53-48a9-81a9-9b036336d428 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:09.306244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef18ec9-3d53-48a9-81a9-9b036336d428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:09.328848 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-fef18ec9-3d53-48a9-81a9-9b036336d428 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:09.330136 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-fef18ec9-3d53-48a9-81a9-9b036336d428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:09.341932 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 886/1752] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:06:09.343126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef18ec9-3d53-48a9-81a9-9b036336d428 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:09.349250 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:09.351312 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5da56577-94e2-40af-8d78-67eb74481df6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7457566282144956bff50037d25737dc', 'request_id': 'req-5da56577-94e2-40af-8d78-67eb74481df6'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:09.352844 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef18ec9-3d53-48a9-81a9-9b036336d428 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:09.357378 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5da56577-94e2-40af-8d78-67eb74481df6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7457566282144956bff50037d25737dc', 'request_id': 'req-5da56577-94e2-40af-8d78-67eb74481df6'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:09.365585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5da56577-94e2-40af-8d78-67eb74481df6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7457566282144956bff50037d25737dc', 'request_id': 'req-5da56577-94e2-40af-8d78-67eb74481df6'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:09.379128 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5da56577-94e2-40af-8d78-67eb74481df6 admin admin] The token cache is being invalidate because project 7457566282144956bff50037d25737dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:06:09.379475 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5da56577-94e2-40af-8d78-67eb74481df6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:09.380430 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5da56577-94e2-40af-8d78-67eb74481df6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:09.382785 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 867/1753] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:06:09 2022] DELETE /identity/v3/projects/7457566282144956bff50037d25737dc => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:09.389038 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3d3f9-9319-4e79-9b90-5681f48a4f19 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:09.389174 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3d3f9-9319-4e79-9b90-5681f48a4f19 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:09.389363 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3d3f9-9319-4e79-9b90-5681f48a4f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:09.410019 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-ebc3d3f9-9319-4e79-9b90-5681f48a4f19 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:09.411756 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ebc3d3f9-9319-4e79-9b90-5681f48a4f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:09.421896 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa049506-e7b6-4779-a20e-35a73b88f92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:09.422521 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa049506-e7b6-4779-a20e-35a73b88f92b admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:09.422792 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa049506-e7b6-4779-a20e-35a73b88f92b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:09.423064 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa049506-e7b6-4779-a20e-35a73b88f92b admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:09.423709 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa049506-e7b6-4779-a20e-35a73b88f92b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:06:09.424042 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa049506-e7b6-4779-a20e-35a73b88f92b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:09.425657 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa049506-e7b6-4779-a20e-35a73b88f92b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:09.438567 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 887/1754] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 06:06:09.442782 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa049506-e7b6-4779-a20e-35a73b88f92b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:09.444479 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-aa049506-e7b6-4779-a20e-35a73b88f92b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:09.456534 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa049506-e7b6-4779-a20e-35a73b88f92b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:09.461192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642e8c2-956d-4949-91fc-e679386ae3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:09.461789 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642e8c2-956d-4949-91fc-e679386ae3b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:09.462065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642e8c2-956d-4949-91fc-e679386ae3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:09.462339 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642e8c2-956d-4949-91fc-e679386ae3b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:09.462953 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5642e8c2-956d-4949-91fc-e679386ae3b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:09.464868 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5642e8c2-956d-4949-91fc-e679386ae3b9 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:09.484067 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 868/1755] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:06:09.493478 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c55bf-7f72-4b3f-a48e-1250e1797574 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:09.493773 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c55bf-7f72-4b3f-a48e-1250e1797574 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:09.494026 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c55bf-7f72-4b3f-a48e-1250e1797574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:09.496817 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 888/1756] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:06:09 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:06:09.498398 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642e8c2-956d-4949-91fc-e679386ae3b9 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:09.514240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-9c5c55bf-7f72-4b3f-a48e-1250e1797574 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:09.515621 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-9c5c55bf-7f72-4b3f-a48e-1250e1797574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:09.570783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642e8c2-956d-4949-91fc-e679386ae3b9 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:09.576399 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3dfdf-cc58-47ce-9015-5c570201ad11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:09.577054 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3dfdf-cc58-47ce-9015-5c570201ad11 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:09.577336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3dfdf-cc58-47ce-9015-5c570201ad11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:09.577641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3dfdf-cc58-47ce-9015-5c570201ad11 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:09.578264 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3dfdf-cc58-47ce-9015-5c570201ad11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:09.580618 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3dfdf-cc58-47ce-9015-5c570201ad11 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:09.591207 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 869/1757] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:06:09.595680 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c55bf-7f72-4b3f-a48e-1250e1797574 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:09.603659 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 889/1758] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:06:09 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:06:09.605888 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c55bf-7f72-4b3f-a48e-1250e1797574 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:09.607242 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3dfdf-cc58-47ce-9015-5c570201ad11 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:09.610771 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c13ef6-379e-48e0-adf9-9d370a5b1906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:09.611402 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c13ef6-379e-48e0-adf9-9d370a5b1906 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:09.611732 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c13ef6-379e-48e0-adf9-9d370a5b1906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:09.612046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c13ef6-379e-48e0-adf9-9d370a5b1906 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:09.612742 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c13ef6-379e-48e0-adf9-9d370a5b1906 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:06:09.613050 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c13ef6-379e-48e0-adf9-9d370a5b1906 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:09.615309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c13ef6-379e-48e0-adf9-9d370a5b1906 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:09.617567 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3dfdf-cc58-47ce-9015-5c570201ad11 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:09.617955 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:09.622713 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e96116-d750-4628-a092-6873f8d6b0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:09.623314 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e96116-d750-4628-a092-6873f8d6b0ea admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:09.623596 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e96116-d750-4628-a092-6873f8d6b0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:09.623885 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e96116-d750-4628-a092-6873f8d6b0ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:09.624723 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e96116-d750-4628-a092-6873f8d6b0ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:09.626451 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e96116-d750-4628-a092-6873f8d6b0ea admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:09.633080 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 870/1759] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 06:06:09.634050 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c13ef6-379e-48e0-adf9-9d370a5b1906 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:09.637807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 890/1760] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:06:09.641189 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e96116-d750-4628-a092-6873f8d6b0ea admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:09.643946 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c13ef6-379e-48e0-adf9-9d370a5b1906 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:09.648758 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab07076-ba0f-423b-9c9a-4f705f30f90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:09.649346 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab07076-ba0f-423b-9c9a-4f705f30f90f admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:09.649626 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab07076-ba0f-423b-9c9a-4f705f30f90f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:09.649891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab07076-ba0f-423b-9c9a-4f705f30f90f admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:09.650505 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab07076-ba0f-423b-9c9a-4f705f30f90f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:09.650601 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e96116-d750-4628-a092-6873f8d6b0ea admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:09.652387 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab07076-ba0f-423b-9c9a-4f705f30f90f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:09.655087 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0b543-a60c-4ea7-b77f-e3f17913192d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:09.655655 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0b543-a60c-4ea7-b77f-e3f17913192d admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:09.655932 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0b543-a60c-4ea7-b77f-e3f17913192d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:09.656199 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0b543-a60c-4ea7-b77f-e3f17913192d admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:09.656901 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0b543-a60c-4ea7-b77f-e3f17913192d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:09.658455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0b543-a60c-4ea7-b77f-e3f17913192d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:09.683230 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 891/1761] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:06:09.685185 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0b543-a60c-4ea7-b77f-e3f17913192d admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:09.690656 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 871/1762] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:06:09 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:06:09.692106 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab07076-ba0f-423b-9c9a-4f705f30f90f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:09.695440 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0b543-a60c-4ea7-b77f-e3f17913192d admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:09.743780 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:09.761768 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91855071-61a5-432a-ade8-df80b49fa1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:09.762438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91855071-61a5-432a-ade8-df80b49fa1c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:09.762711 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91855071-61a5-432a-ade8-df80b49fa1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:09.762978 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91855071-61a5-432a-ade8-df80b49fa1c8 admin admin] PATH_INFO: `/v3/projects/86bf090b77ac41df88f72dd612ecd543/users/7b58cc2c5999491eb92c004cbc29264a/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:09.765775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab07076-ba0f-423b-9c9a-4f705f30f90f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:09.771478 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706710b1-5dff-4f44-a850-512c5e0f8880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:09.772088 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706710b1-5dff-4f44-a850-512c5e0f8880 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:09.772368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706710b1-5dff-4f44-a850-512c5e0f8880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:09.772644 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706710b1-5dff-4f44-a850-512c5e0f8880 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:09.773303 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706710b1-5dff-4f44-a850-512c5e0f8880 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:09.775231 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706710b1-5dff-4f44-a850-512c5e0f8880 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:09.778043 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91855071-61a5-432a-ade8-df80b49fa1c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86bf090b77ac41df88f72dd612ecd543, user_id=7b58cc2c5999491eb92c004cbc29264a, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:09.780679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91855071-61a5-432a-ade8-df80b49fa1c8 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:09.795626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 892/1763] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:06:09 2022] PUT /identity/v3/projects/86bf090b77ac41df88f72dd612ecd543/users/7b58cc2c5999491eb92c004cbc29264a/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:06:09.798311 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 872/1764] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:06:09 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:06:09.802383 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91855071-61a5-432a-ade8-df80b49fa1c8 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:09.807292 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac380d-c3ed-43b6-bb2e-10262b7259cb None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:09.808312 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac380d-c3ed-43b6-bb2e-10262b7259cb None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:09.808599 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac380d-c3ed-43b6-bb2e-10262b7259cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:09.836954 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-34ac380d-c3ed-43b6-bb2e-10262b7259cb None None] MFA Rules not processed for user `7b58cc2c5999491eb92c004cbc29264a`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:09.838173 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-34ac380d-c3ed-43b6-bb2e-10262b7259cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:09.866072 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91855071-61a5-432a-ade8-df80b49fa1c8 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:09.870733 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272c3c6-2edf-41a8-8119-4c566b08a9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:09.871350 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272c3c6-2edf-41a8-8119-4c566b08a9cd admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:09.871615 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272c3c6-2edf-41a8-8119-4c566b08a9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:09.871885 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272c3c6-2edf-41a8-8119-4c566b08a9cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:09.872546 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272c3c6-2edf-41a8-8119-4c566b08a9cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:09.874150 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272c3c6-2edf-41a8-8119-4c566b08a9cd admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:09.884940 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 893/1765] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:06:09.889185 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272c3c6-2edf-41a8-8119-4c566b08a9cd admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:09.899742 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272c3c6-2edf-41a8-8119-4c566b08a9cd admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:09.907894 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b24d7-bf77-4e8e-835c-b558dbc14dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:09.908563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b24d7-bf77-4e8e-835c-b558dbc14dbe admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:09.908899 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b24d7-bf77-4e8e-835c-b558dbc14dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:09.909172 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b24d7-bf77-4e8e-835c-b558dbc14dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:09.909817 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b24d7-bf77-4e8e-835c-b558dbc14dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:09.912186 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b24d7-bf77-4e8e-835c-b558dbc14dbe admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:09.916387 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 873/1766] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:06:09.920271 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac380d-c3ed-43b6-bb2e-10262b7259cb None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:09.927972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 894/1767] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:06:09.932434 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b24d7-bf77-4e8e-835c-b558dbc14dbe admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:09.935945 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac380d-c3ed-43b6-bb2e-10262b7259cb None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:09.940816 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88b7ff-4f4e-4b69-bca4-b5578363f262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:09.941436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88b7ff-4f4e-4b69-bca4-b5578363f262 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:09.941717 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88b7ff-4f4e-4b69-bca4-b5578363f262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:09.942012 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88b7ff-4f4e-4b69-bca4-b5578363f262 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:09.942662 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88b7ff-4f4e-4b69-bca4-b5578363f262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:09.943032 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b24d7-bf77-4e8e-835c-b558dbc14dbe admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:09.944639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88b7ff-4f4e-4b69-bca4-b5578363f262 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:09.947835 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1129b5d-a2f8-444c-9424-7ba389e5cd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:09.948502 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1129b5d-a2f8-444c-9424-7ba389e5cd33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:09.948813 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1129b5d-a2f8-444c-9424-7ba389e5cd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:09.949088 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1129b5d-a2f8-444c-9424-7ba389e5cd33 admin admin] PATH_INFO: `/v3/projects/a0bc53d8aac8417ba013a3ad3d357f05/users/370766dee4ae4cd492e48b360f016227/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:09.964893 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1129b5d-a2f8-444c-9424-7ba389e5cd33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0bc53d8aac8417ba013a3ad3d357f05, user_id=370766dee4ae4cd492e48b360f016227, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:09.968250 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1129b5d-a2f8-444c-9424-7ba389e5cd33 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:09.976401 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 874/1768] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:06:09 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:06:09.980950 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88b7ff-4f4e-4b69-bca4-b5578363f262 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:09.984119 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 895/1769] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:06:09 2022] PUT /identity/v3/projects/a0bc53d8aac8417ba013a3ad3d357f05/users/370766dee4ae4cd492e48b360f016227/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:09.994076 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee076eb7-1078-4967-a417-b52937f39332 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:09.994362 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee076eb7-1078-4967-a417-b52937f39332 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:09.994636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee076eb7-1078-4967-a417-b52937f39332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:10.014474 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:10.024102 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-ee076eb7-1078-4967-a417-b52937f39332 None None] MFA Rules not processed for user `370766dee4ae4cd492e48b360f016227`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:10.025644 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ee076eb7-1078-4967-a417-b52937f39332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:10.048130 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88b7ff-4f4e-4b69-bca4-b5578363f262 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:10.099334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 896/1770] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:06:10.109999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee076eb7-1078-4967-a417-b52937f39332 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:10.110807 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b529390-9f63-4627-b6ff-1734dbc88367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:10.111334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ee076eb7-1078-4967-a417-b52937f39332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:10.111435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b529390-9f63-4627-b6ff-1734dbc88367 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:10.111701 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b529390-9f63-4627-b6ff-1734dbc88367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:10.111973 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b529390-9f63-4627-b6ff-1734dbc88367 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:10.112594 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b529390-9f63-4627-b6ff-1734dbc88367 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:10.114688 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b529390-9f63-4627-b6ff-1734dbc88367 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:10.131550 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:10.135880 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 875/1771] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:06:09 2022] POST /identity/v3/users => generated 296 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:06:10.139879 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b529390-9f63-4627-b6ff-1734dbc88367 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:10.150111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b529390-9f63-4627-b6ff-1734dbc88367 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:10.155128 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe38a4f-efc4-440e-8d1b-1eced544e349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:10.155722 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe38a4f-efc4-440e-8d1b-1eced544e349 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:10.156003 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe38a4f-efc4-440e-8d1b-1eced544e349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:10.156273 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe38a4f-efc4-440e-8d1b-1eced544e349 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:10.156948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe38a4f-efc4-440e-8d1b-1eced544e349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:10.158520 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe38a4f-efc4-440e-8d1b-1eced544e349 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:10.170038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee076eb7-1078-4967-a417-b52937f39332 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:10.172091 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 876/1772] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:06:10.173148 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe38a4f-efc4-440e-8d1b-1eced544e349 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:10.175637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734fb9d-7151-4c17-be8d-fb420d7e0c49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:10.176278 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734fb9d-7151-4c17-be8d-fb420d7e0c49 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:10.176562 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734fb9d-7151-4c17-be8d-fb420d7e0c49 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:10.176996 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734fb9d-7151-4c17-be8d-fb420d7e0c49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:10.182826 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe38a4f-efc4-440e-8d1b-1eced544e349 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:10.185935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734fb9d-7151-4c17-be8d-fb420d7e0c49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:10.187352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734fb9d-7151-4c17-be8d-fb420d7e0c49 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:10.187534 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7548c-5b53-45ab-a861-695532a12f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:10.188173 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7548c-5b53-45ab-a861-695532a12f03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:10.188452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7548c-5b53-45ab-a861-695532a12f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:10.188771 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7548c-5b53-45ab-a861-695532a12f03 admin admin] PATH_INFO: `/v3/projects/e2343457cb1d4276bcee749ebf788e92/users/7fa421b90c94422fa3172249dbf77fea/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:10.197598 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 897/1773] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:06:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:06:10.206503 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7548c-5b53-45ab-a861-695532a12f03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2343457cb1d4276bcee749ebf788e92, user_id=7fa421b90c94422fa3172249dbf77fea, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:10.209888 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7548c-5b53-45ab-a861-695532a12f03 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:10.222354 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 877/1774] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:06:10 2022] PUT /identity/v3/projects/e2343457cb1d4276bcee749ebf788e92/users/7fa421b90c94422fa3172249dbf77fea/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:10.226563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734fb9d-7151-4c17-be8d-fb420d7e0c49 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:10.237021 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734fb9d-7151-4c17-be8d-fb420d7e0c49 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:10.309554 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af98042-aa8c-4b6d-ad53-b8f8c9b8e45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:10.310295 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af98042-aa8c-4b6d-ad53-b8f8c9b8e45c admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:10.311128 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af98042-aa8c-4b6d-ad53-b8f8c9b8e45c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:10.311128 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af98042-aa8c-4b6d-ad53-b8f8c9b8e45c admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:10.312136 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af98042-aa8c-4b6d-ad53-b8f8c9b8e45c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:10.314604 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af98042-aa8c-4b6d-ad53-b8f8c9b8e45c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:10.328878 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 898/1775] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:06:10.331038 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7548c-5b53-45ab-a861-695532a12f03 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:10.342959 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7548c-5b53-45ab-a861-695532a12f03 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:10.349243 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af2192-de36-436a-a4b6-c9110def4d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:10.350162 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af2192-de36-436a-a4b6-c9110def4d3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:10.350443 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af2192-de36-436a-a4b6-c9110def4d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:10.350714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af2192-de36-436a-a4b6-c9110def4d3c admin admin] PATH_INFO: `/v3/projects/e2343457cb1d4276bcee749ebf788e92/users/7fa421b90c94422fa3172249dbf77fea/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:10.352844 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af2192-de36-436a-a4b6-c9110def4d3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2343457cb1d4276bcee749ebf788e92, user_id=7fa421b90c94422fa3172249dbf77fea, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:10.355534 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af2192-de36-436a-a4b6-c9110def4d3c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:10.366961 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 878/1776] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:06:10 2022] PUT /identity/v3/projects/e2343457cb1d4276bcee749ebf788e92/users/7fa421b90c94422fa3172249dbf77fea/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:10.377422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511a438-8e7f-4b30-b290-ec5eb1b429a5 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:10.377712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511a438-8e7f-4b30-b290-ec5eb1b429a5 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:10.377989 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511a438-8e7f-4b30-b290-ec5eb1b429a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:10.414685 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-9511a438-8e7f-4b30-b290-ec5eb1b429a5 None None] MFA Rules not processed for user `7fa421b90c94422fa3172249dbf77fea`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:10.416049 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9511a438-8e7f-4b30-b290-ec5eb1b429a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:10.522471 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 899/1777] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:10 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:06:10.535125 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af2192-de36-436a-a4b6-c9110def4d3c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:10.610972 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af2192-de36-436a-a4b6-c9110def4d3c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:10.623790 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb554b-28f9-4158-96b6-3bf0321180d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:10.624434 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb554b-28f9-4158-96b6-3bf0321180d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:10.624742 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb554b-28f9-4158-96b6-3bf0321180d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:10.625025 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb554b-28f9-4158-96b6-3bf0321180d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:10.638999 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb554b-28f9-4158-96b6-3bf0321180d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:10.638999 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb554b-28f9-4158-96b6-3bf0321180d2 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:10.648751 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 879/1778] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:06:10 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:06:11.005266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511a438-8e7f-4b30-b290-ec5eb1b429a5 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:11.014761 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511a438-8e7f-4b30-b290-ec5eb1b429a5 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:11.021603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880aa83-5c5a-4d76-9afd-58d0bd726117 tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fa421b90c94422fa3172249dbf77fea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2343457cb1d4276bcee749ebf788e92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:11.022238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880aa83-5c5a-4d76-9afd-58d0bd726117 tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:11.022526 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880aa83-5c5a-4d76-9afd-58d0bd726117 tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:11.022969 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880aa83-5c5a-4d76-9afd-58d0bd726117 tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:11.023862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880aa83-5c5a-4d76-9afd-58d0bd726117 tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:06:11.024291 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880aa83-5c5a-4d76-9afd-58d0bd726117 tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:11.026048 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880aa83-5c5a-4d76-9afd-58d0bd726117 tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:11.039136 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 900/1779] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:06:11.043376 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb554b-28f9-4158-96b6-3bf0321180d2 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:11.052585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb554b-28f9-4158-96b6-3bf0321180d2 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:11.057260 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e1c3c-5596-48ac-bdbc-7e66d0fbf95d tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fa421b90c94422fa3172249dbf77fea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2343457cb1d4276bcee749ebf788e92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:11.057883 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e1c3c-5596-48ac-bdbc-7e66d0fbf95d tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:11.058165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e1c3c-5596-48ac-bdbc-7e66d0fbf95d tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:11.058444 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e1c3c-5596-48ac-bdbc-7e66d0fbf95d tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:11.059070 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e1c3c-5596-48ac-bdbc-7e66d0fbf95d tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:11.061054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e1c3c-5596-48ac-bdbc-7e66d0fbf95d tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:11.087858 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:11.098244 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 880/1780] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:06:11 2022] POST /identity/v3/projects => generated 354 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:06:11.099764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880aa83-5c5a-4d76-9afd-58d0bd726117 tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:11.109914 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880aa83-5c5a-4d76-9afd-58d0bd726117 tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:11.136006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:11.170472 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb96c5-ba51-4a26-b16f-d6cbb0b0cbdb tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fa421b90c94422fa3172249dbf77fea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2343457cb1d4276bcee749ebf788e92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:11.171081 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb96c5-ba51-4a26-b16f-d6cbb0b0cbdb tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:11.171359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb96c5-ba51-4a26-b16f-d6cbb0b0cbdb tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:11.171634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb96c5-ba51-4a26-b16f-d6cbb0b0cbdb tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:11.172272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb96c5-ba51-4a26-b16f-d6cbb0b0cbdb tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:06:11.172577 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb96c5-ba51-4a26-b16f-d6cbb0b0cbdb tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:11.174211 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb96c5-ba51-4a26-b16f-d6cbb0b0cbdb tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:11.189671 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 901/1781] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:06:11.226870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e1c3c-5596-48ac-bdbc-7e66d0fbf95d tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:11.238433 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e1c3c-5596-48ac-bdbc-7e66d0fbf95d tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:11.244153 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c000f7-c549-44ad-bcd0-64585ceddae6 tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fa421b90c94422fa3172249dbf77fea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2343457cb1d4276bcee749ebf788e92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:11.244833 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c000f7-c549-44ad-bcd0-64585ceddae6 tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:11.245127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c000f7-c549-44ad-bcd0-64585ceddae6 tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:11.245420 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c000f7-c549-44ad-bcd0-64585ceddae6 tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] PATH_INFO: `/v3/projects/3ef593f7a7b741b9b5c35d05f02c171e` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:11.246535 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c000f7-c549-44ad-bcd0-64585ceddae6 tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3ef593f7a7b741b9b5c35d05f02c171e)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:11.248472 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c000f7-c549-44ad-bcd0-64585ceddae6 tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:11.303440 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-31c000f7-c549-44ad-bcd0-64585ceddae6 tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ef593f7a7b741b9b5c35d05f02c171e', 'request_id': 'req-31c000f7-c549-44ad-bcd0-64585ceddae6'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:11.310637 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-31c000f7-c549-44ad-bcd0-64585ceddae6 tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ef593f7a7b741b9b5c35d05f02c171e', 'request_id': 'req-31c000f7-c549-44ad-bcd0-64585ceddae6'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:11.318613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-31c000f7-c549-44ad-bcd0-64585ceddae6 tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ef593f7a7b741b9b5c35d05f02c171e', 'request_id': 'req-31c000f7-c549-44ad-bcd0-64585ceddae6'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:11.331942 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-31c000f7-c549-44ad-bcd0-64585ceddae6 tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] The token cache is being invalidate because project 3ef593f7a7b741b9b5c35d05f02c171e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:06:11.332234 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-31c000f7-c549-44ad-bcd0-64585ceddae6 tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:11.333997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-31c000f7-c549-44ad-bcd0-64585ceddae6 tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:11.335521 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 881/1782] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:06:11 2022] DELETE /identity/v3/projects/3ef593f7a7b741b9b5c35d05f02c171e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:06:11.551355 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb96c5-ba51-4a26-b16f-d6cbb0b0cbdb tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:11.553241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e1eb96c5-ba51-4a26-b16f-d6cbb0b0cbdb tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:11.629140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb96c5-ba51-4a26-b16f-d6cbb0b0cbdb tempest-VolumeQuotaClassesTest-661210390 tempest-VolumeQuotaClassesTest-661210390-project-admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:11.685704 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a88d15-9ac0-4376-9acd-f6b7612880e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:11.686345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a88d15-9ac0-4376-9acd-f6b7612880e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:11.686623 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a88d15-9ac0-4376-9acd-f6b7612880e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:11.686899 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a88d15-9ac0-4376-9acd-f6b7612880e3 admin admin] PATH_INFO: `/v3/users/7b58cc2c5999491eb92c004cbc29264a` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:11.688034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a88d15-9ac0-4376-9acd-f6b7612880e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b58cc2c5999491eb92c004cbc29264a)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:11.689966 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a88d15-9ac0-4376-9acd-f6b7612880e3 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:11.748934 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-89a88d15-9ac0-4376-9acd-f6b7612880e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b58cc2c5999491eb92c004cbc29264a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:11.748934 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-89a88d15-9ac0-4376-9acd-f6b7612880e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b58cc2c5999491eb92c004cbc29264a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:11.757347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-89a88d15-9ac0-4376-9acd-f6b7612880e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b58cc2c5999491eb92c004cbc29264a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:11.773975 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-89a88d15-9ac0-4376-9acd-f6b7612880e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b58cc2c5999491eb92c004cbc29264a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:11.774926 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-89a88d15-9ac0-4376-9acd-f6b7612880e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b58cc2c5999491eb92c004cbc29264a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:11.776889 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:11.792247 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 902/1783] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:06:11 2022] DELETE /identity/v3/users/7b58cc2c5999491eb92c004cbc29264a => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:11.796954 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d54e4-eee3-4952-9cbf-5268b13b3e43 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:11.798025 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d54e4-eee3-4952-9cbf-5268b13b3e43 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:11.798451 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d54e4-eee3-4952-9cbf-5268b13b3e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:11.824523 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-ff6d54e4-eee3-4952-9cbf-5268b13b3e43 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:11.825795 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ff6d54e4-eee3-4952-9cbf-5268b13b3e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:11.902676 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 882/1784] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:06:11.913322 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a88d15-9ac0-4376-9acd-f6b7612880e3 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:11.914781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-89a88d15-9ac0-4376-9acd-f6b7612880e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:11.980216 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a88d15-9ac0-4376-9acd-f6b7612880e3 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:11.984371 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a53180-58a6-42c8-8e24-ad500979c33b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:11.984948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a53180-58a6-42c8-8e24-ad500979c33b service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:11.985227 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a53180-58a6-42c8-8e24-ad500979c33b service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:11.985493 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a53180-58a6-42c8-8e24-ad500979c33b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:12.005575 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a53180-58a6-42c8-8e24-ad500979c33b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:12.007001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a53180-58a6-42c8-8e24-ad500979c33b service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:12.019092 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 903/1785] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:06:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:06:12.306279 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d54e4-eee3-4952-9cbf-5268b13b3e43 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:12.309505 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ff6d54e4-eee3-4952-9cbf-5268b13b3e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:12.320886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d54e4-eee3-4952-9cbf-5268b13b3e43 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:12.325601 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3dd47-0747-49e2-b6e9-40972394118a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:12.326237 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3dd47-0747-49e2-b6e9-40972394118a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:12.326552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3dd47-0747-49e2-b6e9-40972394118a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:12.326782 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3dd47-0747-49e2-b6e9-40972394118a admin admin] PATH_INFO: `/v3/projects/86bf090b77ac41df88f72dd612ecd543` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:12.327896 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac3dd47-0747-49e2-b6e9-40972394118a admin admin] RBAC: Authorizing `identity:delete_project(project_id=86bf090b77ac41df88f72dd612ecd543)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:12.329854 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac3dd47-0747-49e2-b6e9-40972394118a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:12.380344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fac3dd47-0747-49e2-b6e9-40972394118a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86bf090b77ac41df88f72dd612ecd543', 'request_id': 'req-fac3dd47-0747-49e2-b6e9-40972394118a'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:12.386926 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fac3dd47-0747-49e2-b6e9-40972394118a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86bf090b77ac41df88f72dd612ecd543', 'request_id': 'req-fac3dd47-0747-49e2-b6e9-40972394118a'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:12.394379 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fac3dd47-0747-49e2-b6e9-40972394118a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86bf090b77ac41df88f72dd612ecd543', 'request_id': 'req-fac3dd47-0747-49e2-b6e9-40972394118a'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:12.409165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fac3dd47-0747-49e2-b6e9-40972394118a admin admin] The token cache is being invalidate because project 86bf090b77ac41df88f72dd612ecd543 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:06:12.409474 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fac3dd47-0747-49e2-b6e9-40972394118a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:12.410154 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fac3dd47-0747-49e2-b6e9-40972394118a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:12.412535 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 883/1786] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:06:12 2022] DELETE /identity/v3/projects/86bf090b77ac41df88f72dd612ecd543 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:12.416307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a53180-58a6-42c8-8e24-ad500979c33b service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:12.426044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a53180-58a6-42c8-8e24-ad500979c33b service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:12.482632 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58ef5b-b8c5-4fb7-b9b7-0115dd6bae6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:12.483180 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58ef5b-b8c5-4fb7-b9b7-0115dd6bae6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:12.483462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58ef5b-b8c5-4fb7-b9b7-0115dd6bae6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:12.483738 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58ef5b-b8c5-4fb7-b9b7-0115dd6bae6b admin admin] PATH_INFO: `/v3/users/7fa421b90c94422fa3172249dbf77fea` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:12.484905 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58ef5b-b8c5-4fb7-b9b7-0115dd6bae6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fa421b90c94422fa3172249dbf77fea)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:12.486883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58ef5b-b8c5-4fb7-b9b7-0115dd6bae6b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:12.530966 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ad58ef5b-b8c5-4fb7-b9b7-0115dd6bae6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fa421b90c94422fa3172249dbf77fea'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:12.531595 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ad58ef5b-b8c5-4fb7-b9b7-0115dd6bae6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fa421b90c94422fa3172249dbf77fea'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:12.539538 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ad58ef5b-b8c5-4fb7-b9b7-0115dd6bae6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fa421b90c94422fa3172249dbf77fea'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:12.552038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ad58ef5b-b8c5-4fb7-b9b7-0115dd6bae6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fa421b90c94422fa3172249dbf77fea'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:12.552812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ad58ef5b-b8c5-4fb7-b9b7-0115dd6bae6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fa421b90c94422fa3172249dbf77fea'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:12.564635 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 904/1787] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:06:12 2022] DELETE /identity/v3/users/7fa421b90c94422fa3172249dbf77fea => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:12.833452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3dd47-0747-49e2-b6e9-40972394118a admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:12.834900 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fac3dd47-0747-49e2-b6e9-40972394118a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:12.847911 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3dd47-0747-49e2-b6e9-40972394118a admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:12.852631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:12.908570 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6d01b-bc79-41ee-a419-41fdd83ed804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:12.909257 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6d01b-bc79-41ee-a419-41fdd83ed804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:12.909542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6d01b-bc79-41ee-a419-41fdd83ed804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:12.909815 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6d01b-bc79-41ee-a419-41fdd83ed804 admin admin] PATH_INFO: `/v3/projects/e2343457cb1d4276bcee749ebf788e92` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:12.911640 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6d01b-bc79-41ee-a419-41fdd83ed804 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2343457cb1d4276bcee749ebf788e92)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:12.913593 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6d01b-bc79-41ee-a419-41fdd83ed804 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:12.962875 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-dbe6d01b-bc79-41ee-a419-41fdd83ed804 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2343457cb1d4276bcee749ebf788e92', 'request_id': 'req-dbe6d01b-bc79-41ee-a419-41fdd83ed804'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:12.968605 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-dbe6d01b-bc79-41ee-a419-41fdd83ed804 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2343457cb1d4276bcee749ebf788e92', 'request_id': 'req-dbe6d01b-bc79-41ee-a419-41fdd83ed804'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:12.975921 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-dbe6d01b-bc79-41ee-a419-41fdd83ed804 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2343457cb1d4276bcee749ebf788e92', 'request_id': 'req-dbe6d01b-bc79-41ee-a419-41fdd83ed804'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:12.990238 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-dbe6d01b-bc79-41ee-a419-41fdd83ed804 admin admin] The token cache is being invalidate because project e2343457cb1d4276bcee749ebf788e92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:06:12.990573 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-dbe6d01b-bc79-41ee-a419-41fdd83ed804 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:12.991145 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-dbe6d01b-bc79-41ee-a419-41fdd83ed804 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:12.992645 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 884/1788] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:06:12 2022] DELETE /identity/v3/projects/e2343457cb1d4276bcee749ebf788e92 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:12.997538 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a14348-b3b9-46a4-a59c-f9b51b6d9391 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:12.997823 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a14348-b3b9-46a4-a59c-f9b51b6d9391 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:12.998094 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a14348-b3b9-46a4-a59c-f9b51b6d9391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:13.018047 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-54a14348-b3b9-46a4-a59c-f9b51b6d9391 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:13.019112 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-54a14348-b3b9-46a4-a59c-f9b51b6d9391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:13.024845 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:13.088668 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 905/1789] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:06:13.096673 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441db06f-516f-4c47-ab03-c5ac1fcdac4b None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:13.096982 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441db06f-516f-4c47-ab03-c5ac1fcdac4b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:13.097249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441db06f-516f-4c47-ab03-c5ac1fcdac4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:13.120192 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-441db06f-516f-4c47-ab03-c5ac1fcdac4b None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:13.133129 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-441db06f-516f-4c47-ab03-c5ac1fcdac4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:13.133129 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 885/1790] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:06:13.138898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a14348-b3b9-46a4-a59c-f9b51b6d9391 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:13.149160 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a14348-b3b9-46a4-a59c-f9b51b6d9391 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:13.153767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a16f98-99b2-48b2-a433-c11c15b0e667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:13.154370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a16f98-99b2-48b2-a433-c11c15b0e667 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:13.154644 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a16f98-99b2-48b2-a433-c11c15b0e667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:13.155083 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a16f98-99b2-48b2-a433-c11c15b0e667 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:13.155734 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a16f98-99b2-48b2-a433-c11c15b0e667 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:06:13.156039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a16f98-99b2-48b2-a433-c11c15b0e667 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:13.157665 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a16f98-99b2-48b2-a433-c11c15b0e667 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:13.174042 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 906/1791] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:06:13.175172 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441db06f-516f-4c47-ab03-c5ac1fcdac4b None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:13.184577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441db06f-516f-4c47-ab03-c5ac1fcdac4b None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:13.189232 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073727e-fc27-4c62-ad24-f465155881fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:13.189821 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073727e-fc27-4c62-ad24-f465155881fc admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:13.190093 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073727e-fc27-4c62-ad24-f465155881fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:13.190359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073727e-fc27-4c62-ad24-f465155881fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:13.190974 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073727e-fc27-4c62-ad24-f465155881fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:13.193404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073727e-fc27-4c62-ad24-f465155881fc admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:13.220588 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 886/1792] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:06:13 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:06:13.225151 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a16f98-99b2-48b2-a433-c11c15b0e667 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:13.235242 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a16f98-99b2-48b2-a433-c11c15b0e667 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:13.291870 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4843015-2eba-4384-84bb-75d9f6eb98f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:13.292517 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4843015-2eba-4384-84bb-75d9f6eb98f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:13.292907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4843015-2eba-4384-84bb-75d9f6eb98f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:13.293099 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4843015-2eba-4384-84bb-75d9f6eb98f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:13.293723 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4843015-2eba-4384-84bb-75d9f6eb98f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:13.298896 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4843015-2eba-4384-84bb-75d9f6eb98f5 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:13.323494 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 907/1793] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:06:13 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:06:13.327548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073727e-fc27-4c62-ad24-f465155881fc admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:13.339458 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073727e-fc27-4c62-ad24-f465155881fc admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:13.345210 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29dfdb3-a59f-453e-bca9-ed78bd4d1275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:13.346320 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29dfdb3-a59f-453e-bca9-ed78bd4d1275 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:13.346600 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29dfdb3-a59f-453e-bca9-ed78bd4d1275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:13.346891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29dfdb3-a59f-453e-bca9-ed78bd4d1275 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:13.348996 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29dfdb3-a59f-453e-bca9-ed78bd4d1275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:13.350638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29dfdb3-a59f-453e-bca9-ed78bd4d1275 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:13.361334 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 887/1794] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:06:13.365042 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4843015-2eba-4384-84bb-75d9f6eb98f5 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:13.375016 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4843015-2eba-4384-84bb-75d9f6eb98f5 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:13.380637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae16ab5-2810-448a-bec3-5ade8314c393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:13.381269 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae16ab5-2810-448a-bec3-5ade8314c393 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:13.381571 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae16ab5-2810-448a-bec3-5ade8314c393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:13.381865 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae16ab5-2810-448a-bec3-5ade8314c393 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:13.382621 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae16ab5-2810-448a-bec3-5ade8314c393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:13.384721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae16ab5-2810-448a-bec3-5ade8314c393 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:13.397439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 908/1795] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:06:13.403257 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29dfdb3-a59f-453e-bca9-ed78bd4d1275 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:13.416875 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29dfdb3-a59f-453e-bca9-ed78bd4d1275 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:13.418823 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35e2f4-92c6-4f51-95cd-6bfc73a8929e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:13.419559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35e2f4-92c6-4f51-95cd-6bfc73a8929e admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:13.419908 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35e2f4-92c6-4f51-95cd-6bfc73a8929e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:13.420251 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35e2f4-92c6-4f51-95cd-6bfc73a8929e admin admin] PATH_INFO: `/v3/projects/7b0d38ca8c254eb4a48521422cfed830/users/19d4b11cfa5d4c8ca31463ab1a54d61f/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:13.440401 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35e2f4-92c6-4f51-95cd-6bfc73a8929e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b0d38ca8c254eb4a48521422cfed830, user_id=19d4b11cfa5d4c8ca31463ab1a54d61f, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:13.444009 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35e2f4-92c6-4f51-95cd-6bfc73a8929e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:13.459378 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 888/1796] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:06:13 2022] PUT /identity/v3/projects/7b0d38ca8c254eb4a48521422cfed830/users/19d4b11cfa5d4c8ca31463ab1a54d61f/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:13.464607 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c49db8-4886-4c2d-9c10-35fb7fb82ba9 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:13.465076 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c49db8-4886-4c2d-9c10-35fb7fb82ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:13.465270 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c49db8-4886-4c2d-9c10-35fb7fb82ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:13.488336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-e1c49db8-4886-4c2d-9c10-35fb7fb82ba9 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:13.489760 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e1c49db8-4886-4c2d-9c10-35fb7fb82ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:13.550439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 909/1797] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:06:13.560249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35e2f4-92c6-4f51-95cd-6bfc73a8929e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:13.561987 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2f35e2f4-92c6-4f51-95cd-6bfc73a8929e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:13.597719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:13.620811 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35e2f4-92c6-4f51-95cd-6bfc73a8929e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:13.624496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5dac6f-0dd4-417d-803b-004b852fbfa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:13.625133 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5dac6f-0dd4-417d-803b-004b852fbfa4 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:13.625410 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5dac6f-0dd4-417d-803b-004b852fbfa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:13.625684 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5dac6f-0dd4-417d-803b-004b852fbfa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:13.636653 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5dac6f-0dd4-417d-803b-004b852fbfa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:13.638134 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5dac6f-0dd4-417d-803b-004b852fbfa4 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:13.648790 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 889/1798] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:06:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:06:14.258286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb4645-b446-4bc0-a4f4-195d6fb1690e None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:14.259075 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb4645-b446-4bc0-a4f4-195d6fb1690e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:14.259182 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb4645-b446-4bc0-a4f4-195d6fb1690e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:14.289174 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-2fdb4645-b446-4bc0-a4f4-195d6fb1690e None None] MFA Rules not processed for user `19d4b11cfa5d4c8ca31463ab1a54d61f`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:14.291017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2fdb4645-b446-4bc0-a4f4-195d6fb1690e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:14.301802 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:14.383269 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 910/1799] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:06:14.388134 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5dac6f-0dd4-417d-803b-004b852fbfa4 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:14.398278 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5dac6f-0dd4-417d-803b-004b852fbfa4 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:14.403004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245f632-c4f8-4ee6-92f8-a70483fde2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:14.403603 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245f632-c4f8-4ee6-92f8-a70483fde2ea admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:14.403883 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245f632-c4f8-4ee6-92f8-a70483fde2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:14.404157 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245f632-c4f8-4ee6-92f8-a70483fde2ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:14.404827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245f632-c4f8-4ee6-92f8-a70483fde2ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:14.406723 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245f632-c4f8-4ee6-92f8-a70483fde2ea admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:14.434185 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 890/1800] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:06:14 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:06:14.438478 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb4645-b446-4bc0-a4f4-195d6fb1690e None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:14.450677 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb4645-b446-4bc0-a4f4-195d6fb1690e None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:14.509038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728f5e5-793a-4093-9c83-c6dd9c75b33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:14.509647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728f5e5-793a-4093-9c83-c6dd9c75b33a admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:14.509926 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728f5e5-793a-4093-9c83-c6dd9c75b33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:14.510192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728f5e5-793a-4093-9c83-c6dd9c75b33a admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:14.510835 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728f5e5-793a-4093-9c83-c6dd9c75b33a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:14.512755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728f5e5-793a-4093-9c83-c6dd9c75b33a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:14.534603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 911/1801] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:06:14 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:06:14.538743 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245f632-c4f8-4ee6-92f8-a70483fde2ea admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:14.549440 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245f632-c4f8-4ee6-92f8-a70483fde2ea admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:14.555013 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e2ca0-7ba2-40df-8473-6cc084f59c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:14.555617 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e2ca0-7ba2-40df-8473-6cc084f59c4e admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:14.555894 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e2ca0-7ba2-40df-8473-6cc084f59c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:14.556165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e2ca0-7ba2-40df-8473-6cc084f59c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:14.556834 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e2ca0-7ba2-40df-8473-6cc084f59c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:14.558420 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e2ca0-7ba2-40df-8473-6cc084f59c4e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:14.569202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 891/1802] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:06:14.573626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728f5e5-793a-4093-9c83-c6dd9c75b33a admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:14.592868 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728f5e5-793a-4093-9c83-c6dd9c75b33a admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:14.592868 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612d251-1da9-4133-9b81-73ffedbd3493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:14.592868 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612d251-1da9-4133-9b81-73ffedbd3493 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:14.592868 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612d251-1da9-4133-9b81-73ffedbd3493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:14.593241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612d251-1da9-4133-9b81-73ffedbd3493 admin admin] PATH_INFO: `/v3/projects/82bad4f651b24ea7876d4723a0666eac/users/c684b50e5f184065aaebbe99b27836e9/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:14.606663 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3612d251-1da9-4133-9b81-73ffedbd3493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82bad4f651b24ea7876d4723a0666eac, user_id=c684b50e5f184065aaebbe99b27836e9, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:14.609507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3612d251-1da9-4133-9b81-73ffedbd3493 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:14.622972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 912/1803] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:06:14 2022] PUT /identity/v3/projects/82bad4f651b24ea7876d4723a0666eac/users/c684b50e5f184065aaebbe99b27836e9/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:06:14.626758 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e2ca0-7ba2-40df-8473-6cc084f59c4e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:14.638308 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e2ca0-7ba2-40df-8473-6cc084f59c4e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:14.697626 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a6613-f8cd-43ba-969c-ac60c4adf46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:14.699309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a6613-f8cd-43ba-969c-ac60c4adf46e admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:14.699309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a6613-f8cd-43ba-969c-ac60c4adf46e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:14.699309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a6613-f8cd-43ba-969c-ac60c4adf46e admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:14.701209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a6613-f8cd-43ba-969c-ac60c4adf46e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:14.702998 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a6613-f8cd-43ba-969c-ac60c4adf46e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:14.717088 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 892/1804] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:06:14.721198 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612d251-1da9-4133-9b81-73ffedbd3493 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:14.734811 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612d251-1da9-4133-9b81-73ffedbd3493 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:14.741386 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57db91-91c8-40bc-8f30-bb39729a88dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:14.741967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57db91-91c8-40bc-8f30-bb39729a88dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:14.742247 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57db91-91c8-40bc-8f30-bb39729a88dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:14.742523 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57db91-91c8-40bc-8f30-bb39729a88dc admin admin] PATH_INFO: `/v3/projects/82bad4f651b24ea7876d4723a0666eac/users/c684b50e5f184065aaebbe99b27836e9/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:14.746191 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57db91-91c8-40bc-8f30-bb39729a88dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=82bad4f651b24ea7876d4723a0666eac, user_id=c684b50e5f184065aaebbe99b27836e9, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:14.749081 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57db91-91c8-40bc-8f30-bb39729a88dc admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:14.762615 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 913/1805] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:06:14 2022] PUT /identity/v3/projects/82bad4f651b24ea7876d4723a0666eac/users/c684b50e5f184065aaebbe99b27836e9/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:15.461722 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56795004-3864-4c5b-92ab-371f07178a08 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:15.462730 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56795004-3864-4c5b-92ab-371f07178a08 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:15.462875 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56795004-3864-4c5b-92ab-371f07178a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:15.503765 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-56795004-3864-4c5b-92ab-371f07178a08 None None] MFA Rules not processed for user `c684b50e5f184065aaebbe99b27836e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:15.505566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-56795004-3864-4c5b-92ab-371f07178a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:15.598119 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 893/1806] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:06:15.609494 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57db91-91c8-40bc-8f30-bb39729a88dc admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:15.611283 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ad57db91-91c8-40bc-8f30-bb39729a88dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:15.675853 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57db91-91c8-40bc-8f30-bb39729a88dc admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:15.680256 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30dde0-d8f9-4947-83c5-f97a36eb3ac0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:15.680895 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30dde0-d8f9-4947-83c5-f97a36eb3ac0 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:15.681174 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30dde0-d8f9-4947-83c5-f97a36eb3ac0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:15.681442 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30dde0-d8f9-4947-83c5-f97a36eb3ac0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:15.694361 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:15.741885 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30dde0-d8f9-4947-83c5-f97a36eb3ac0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:15.743350 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30dde0-d8f9-4947-83c5-f97a36eb3ac0 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:15.753691 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 914/1807] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:06:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:06:17.117772 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56795004-3864-4c5b-92ab-371f07178a08 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:17.119192 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-56795004-3864-4c5b-92ab-371f07178a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:17.131834 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56795004-3864-4c5b-92ab-371f07178a08 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:17.135714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e92a05-feea-4e76-b69e-2e43fbd89c55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:17.136309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e92a05-feea-4e76-b69e-2e43fbd89c55 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:17.136585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e92a05-feea-4e76-b69e-2e43fbd89c55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:17.136890 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e92a05-feea-4e76-b69e-2e43fbd89c55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:17.146067 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e92a05-feea-4e76-b69e-2e43fbd89c55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:17.147511 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e92a05-feea-4e76-b69e-2e43fbd89c55 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:17.157140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 894/1808] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:06:17 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:06:20.416225 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30dde0-d8f9-4947-83c5-f97a36eb3ac0 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:20.417849 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5e30dde0-d8f9-4947-83c5-f97a36eb3ac0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:20.492775 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30dde0-d8f9-4947-83c5-f97a36eb3ac0 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:20.497553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a7f7f-093a-4be2-910d-f350dea1d3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:20.498200 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a7f7f-093a-4be2-910d-f350dea1d3a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:20.498447 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a7f7f-093a-4be2-910d-f350dea1d3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:20.498716 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a7f7f-093a-4be2-910d-f350dea1d3a5 admin admin] PATH_INFO: `/v3/users/370766dee4ae4cd492e48b360f016227` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:20.499821 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a7f7f-093a-4be2-910d-f350dea1d3a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=370766dee4ae4cd492e48b360f016227)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:20.501746 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a7f7f-093a-4be2-910d-f350dea1d3a5 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:20.547711 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-679a7f7f-093a-4be2-910d-f350dea1d3a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '370766dee4ae4cd492e48b360f016227'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:20.548348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-679a7f7f-093a-4be2-910d-f350dea1d3a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '370766dee4ae4cd492e48b360f016227'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:20.555162 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-679a7f7f-093a-4be2-910d-f350dea1d3a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '370766dee4ae4cd492e48b360f016227'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:20.567332 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-679a7f7f-093a-4be2-910d-f350dea1d3a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '370766dee4ae4cd492e48b360f016227'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:20.567955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-679a7f7f-093a-4be2-910d-f350dea1d3a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '370766dee4ae4cd492e48b360f016227'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:20.579827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 915/1809] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:06:20 2022] DELETE /identity/v3/users/370766dee4ae4cd492e48b360f016227 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:20.585079 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57193666-1544-45c1-b117-77590b3a0732 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:20.585537 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57193666-1544-45c1-b117-77590b3a0732 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:20.585652 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57193666-1544-45c1-b117-77590b3a0732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:20.604326 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-57193666-1544-45c1-b117-77590b3a0732 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:20.605544 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-57193666-1544-45c1-b117-77590b3a0732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:20.610410 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:20.664093 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 895/1810] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:06:20.673668 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a7f7f-093a-4be2-910d-f350dea1d3a5 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:20.675104 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-679a7f7f-093a-4be2-910d-f350dea1d3a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:20.710862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:20.735095 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a7f7f-093a-4be2-910d-f350dea1d3a5 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:20.738709 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6822ec2e-a546-4d05-967e-53dc6a24886b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:20.739309 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822ec2e-a546-4d05-967e-53dc6a24886b service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:20.739584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822ec2e-a546-4d05-967e-53dc6a24886b service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:20.739922 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822ec2e-a546-4d05-967e-53dc6a24886b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:20.749612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6822ec2e-a546-4d05-967e-53dc6a24886b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:20.751019 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6822ec2e-a546-4d05-967e-53dc6a24886b service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:20.761456 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 916/1811] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:06:21.039004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57193666-1544-45c1-b117-77590b3a0732 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:21.040271 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-57193666-1544-45c1-b117-77590b3a0732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:21.071300 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57193666-1544-45c1-b117-77590b3a0732 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:21.077967 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e466a78-6962-46bb-b4f8-e131af38dd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:21.078836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e466a78-6962-46bb-b4f8-e131af38dd2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:21.079309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e466a78-6962-46bb-b4f8-e131af38dd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:21.079797 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e466a78-6962-46bb-b4f8-e131af38dd2d admin admin] PATH_INFO: `/v3/projects/a0bc53d8aac8417ba013a3ad3d357f05` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:21.081207 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e466a78-6962-46bb-b4f8-e131af38dd2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0bc53d8aac8417ba013a3ad3d357f05)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:21.083411 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e466a78-6962-46bb-b4f8-e131af38dd2d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:21.146536 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5e466a78-6962-46bb-b4f8-e131af38dd2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0bc53d8aac8417ba013a3ad3d357f05', 'request_id': 'req-5e466a78-6962-46bb-b4f8-e131af38dd2d'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:21.152534 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5e466a78-6962-46bb-b4f8-e131af38dd2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0bc53d8aac8417ba013a3ad3d357f05', 'request_id': 'req-5e466a78-6962-46bb-b4f8-e131af38dd2d'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:21.159174 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5e466a78-6962-46bb-b4f8-e131af38dd2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0bc53d8aac8417ba013a3ad3d357f05', 'request_id': 'req-5e466a78-6962-46bb-b4f8-e131af38dd2d'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:21.178097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5e466a78-6962-46bb-b4f8-e131af38dd2d admin admin] The token cache is being invalidate because project a0bc53d8aac8417ba013a3ad3d357f05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:06:21.178517 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5e466a78-6962-46bb-b4f8-e131af38dd2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:21.179317 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5e466a78-6962-46bb-b4f8-e131af38dd2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:21.181174 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 896/1812] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:06:21 2022] DELETE /identity/v3/projects/a0bc53d8aac8417ba013a3ad3d357f05 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:21.186174 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d67db5-f87f-4e55-a7ab-80e2a6cf6466 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:21.186458 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d67db5-f87f-4e55-a7ab-80e2a6cf6466 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:21.186732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d67db5-f87f-4e55-a7ab-80e2a6cf6466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:21.209199 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-23d67db5-f87f-4e55-a7ab-80e2a6cf6466 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:21.210651 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-23d67db5-f87f-4e55-a7ab-80e2a6cf6466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:21.287904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 917/1813] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:06:21.289653 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02559d-75b9-4241-a497-fe53a7a93791 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:21.289953 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02559d-75b9-4241-a497-fe53a7a93791 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:21.290243 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02559d-75b9-4241-a497-fe53a7a93791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:21.328168 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-5f02559d-75b9-4241-a497-fe53a7a93791 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:21.329922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5f02559d-75b9-4241-a497-fe53a7a93791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:21.342961 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 897/1814] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:06:21.346903 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d67db5-f87f-4e55-a7ab-80e2a6cf6466 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:21.357700 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d67db5-f87f-4e55-a7ab-80e2a6cf6466 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:21.363021 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6146b16-539e-4f28-8438-777881eee52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:21.363842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6146b16-539e-4f28-8438-777881eee52c admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:21.364201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6146b16-539e-4f28-8438-777881eee52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:21.366498 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6146b16-539e-4f28-8438-777881eee52c admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:21.366498 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6146b16-539e-4f28-8438-777881eee52c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:06:21.366498 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6146b16-539e-4f28-8438-777881eee52c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:21.367750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6146b16-539e-4f28-8438-777881eee52c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:21.380119 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 918/1815] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:06:21.385171 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f02559d-75b9-4241-a497-fe53a7a93791 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:21.394930 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f02559d-75b9-4241-a497-fe53a7a93791 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:21.400046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822eaaf-0a28-48aa-85b2-81e54a252f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:21.400652 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822eaaf-0a28-48aa-85b2-81e54a252f04 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:21.400976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822eaaf-0a28-48aa-85b2-81e54a252f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:21.401244 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822eaaf-0a28-48aa-85b2-81e54a252f04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:21.401949 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4822eaaf-0a28-48aa-85b2-81e54a252f04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:21.403860 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4822eaaf-0a28-48aa-85b2-81e54a252f04 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:21.433702 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 898/1816] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:06:21 2022] POST /identity/v3/projects => generated 389 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:06:21.438233 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6146b16-539e-4f28-8438-777881eee52c admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:21.450175 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6146b16-539e-4f28-8438-777881eee52c admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:21.513699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc774641-ce7b-48a2-88c5-21516389e99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:21.514497 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc774641-ce7b-48a2-88c5-21516389e99c admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:21.514779 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc774641-ce7b-48a2-88c5-21516389e99c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:21.515050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc774641-ce7b-48a2-88c5-21516389e99c admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:21.515676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc774641-ce7b-48a2-88c5-21516389e99c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:21.517681 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc774641-ce7b-48a2-88c5-21516389e99c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:21.540383 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 919/1817] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:06:21 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:06:21.544493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822eaaf-0a28-48aa-85b2-81e54a252f04 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:21.555214 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822eaaf-0a28-48aa-85b2-81e54a252f04 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:21.559731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a374e-fbc9-4634-a933-0782c1497dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:21.560327 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a374e-fbc9-4634-a933-0782c1497dae admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:21.560602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a374e-fbc9-4634-a933-0782c1497dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:21.560903 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a374e-fbc9-4634-a933-0782c1497dae admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:21.561533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a374e-fbc9-4634-a933-0782c1497dae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:21.563105 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a374e-fbc9-4634-a933-0782c1497dae admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:21.574011 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 899/1818] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:06:21.577996 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc774641-ce7b-48a2-88c5-21516389e99c admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:21.589085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc774641-ce7b-48a2-88c5-21516389e99c admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:21.593712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f47c1a-4841-4d8d-abce-f5612bb7cf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:21.594307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f47c1a-4841-4d8d-abce-f5612bb7cf75 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:21.594589 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f47c1a-4841-4d8d-abce-f5612bb7cf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:21.594862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f47c1a-4841-4d8d-abce-f5612bb7cf75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:21.595494 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f47c1a-4841-4d8d-abce-f5612bb7cf75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:21.597122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f47c1a-4841-4d8d-abce-f5612bb7cf75 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:21.607741 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 920/1819] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:06:21.611819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a374e-fbc9-4634-a933-0782c1497dae admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:21.622047 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a374e-fbc9-4634-a933-0782c1497dae admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:21.626619 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a2b04-ed0b-435e-abec-c9ec7a7e2503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:21.627258 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a2b04-ed0b-435e-abec-c9ec7a7e2503 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:21.627539 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a2b04-ed0b-435e-abec-c9ec7a7e2503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:21.627809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a2b04-ed0b-435e-abec-c9ec7a7e2503 admin admin] PATH_INFO: `/v3/projects/58a000d5925c4ab9a456d2c557c99523/users/ec11700e7fb44eb7ac9475d748c551f0/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:21.642994 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a2b04-ed0b-435e-abec-c9ec7a7e2503 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58a000d5925c4ab9a456d2c557c99523, user_id=ec11700e7fb44eb7ac9475d748c551f0, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:21.645887 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a2b04-ed0b-435e-abec-c9ec7a7e2503 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:21.658716 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 900/1820] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:06:21 2022] PUT /identity/v3/projects/58a000d5925c4ab9a456d2c557c99523/users/ec11700e7fb44eb7ac9475d748c551f0/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:21.669058 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94245ed8-18ab-4a42-9431-44e01136cfea None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:21.669345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94245ed8-18ab-4a42-9431-44e01136cfea None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:21.669614 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94245ed8-18ab-4a42-9431-44e01136cfea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:21.698208 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-94245ed8-18ab-4a42-9431-44e01136cfea None None] MFA Rules not processed for user `ec11700e7fb44eb7ac9475d748c551f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:21.699392 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-94245ed8-18ab-4a42-9431-44e01136cfea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:21.727912 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:21.764747 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 921/1821] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:06:21.775656 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a2b04-ed0b-435e-abec-c9ec7a7e2503 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:21.777403 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-af7a2b04-ed0b-435e-abec-c9ec7a7e2503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:21.811818 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:21.837450 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a2b04-ed0b-435e-abec-c9ec7a7e2503 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:21.841101 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b3759-53a2-4bda-8891-2ce484c324c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:21.841705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b3759-53a2-4bda-8891-2ce484c324c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:21.841982 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b3759-53a2-4bda-8891-2ce484c324c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:21.842255 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b3759-53a2-4bda-8891-2ce484c324c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:21.850601 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b3759-53a2-4bda-8891-2ce484c324c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:21.852012 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b3759-53a2-4bda-8891-2ce484c324c7 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:21.860650 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 901/1822] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:06:21 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:06:31.761694 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=13 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:31.856715 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=12 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:39.793562 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99a133-97a8-49a0-85b2-4d9a2550b837 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:39.794035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99a133-97a8-49a0-85b2-4d9a2550b837 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:39.794180 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99a133-97a8-49a0-85b2-4d9a2550b837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:39.821364 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-9b99a133-97a8-49a0-85b2-4d9a2550b837 None None] MFA Rules not processed for user `2c6be23065c643939055a3de375187d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:39.822573 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9b99a133-97a8-49a0-85b2-4d9a2550b837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:39.886708 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 922/1823] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:06:43.599616 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b3759-53a2-4bda-8891-2ce484c324c7 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:43.601297 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-853b3759-53a2-4bda-8891-2ce484c324c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:43.687598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b3759-53a2-4bda-8891-2ce484c324c7 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:43.692096 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb102e1-e5a7-4c99-889d-42a2ee120d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:43.692737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb102e1-e5a7-4c99-889d-42a2ee120d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:43.693020 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb102e1-e5a7-4c99-889d-42a2ee120d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:43.693289 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb102e1-e5a7-4c99-889d-42a2ee120d2f admin admin] PATH_INFO: `/v3/users/19d4b11cfa5d4c8ca31463ab1a54d61f` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:43.695079 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb102e1-e5a7-4c99-889d-42a2ee120d2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=19d4b11cfa5d4c8ca31463ab1a54d61f)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:43.697095 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb102e1-e5a7-4c99-889d-42a2ee120d2f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:43.759744 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-bdb102e1-e5a7-4c99-889d-42a2ee120d2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19d4b11cfa5d4c8ca31463ab1a54d61f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:43.760455 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-bdb102e1-e5a7-4c99-889d-42a2ee120d2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19d4b11cfa5d4c8ca31463ab1a54d61f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:43.777253 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-bdb102e1-e5a7-4c99-889d-42a2ee120d2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19d4b11cfa5d4c8ca31463ab1a54d61f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:43.790978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-bdb102e1-e5a7-4c99-889d-42a2ee120d2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19d4b11cfa5d4c8ca31463ab1a54d61f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:43.798301 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-bdb102e1-e5a7-4c99-889d-42a2ee120d2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19d4b11cfa5d4c8ca31463ab1a54d61f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:43.804922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 902/1824] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:06:43 2022] DELETE /identity/v3/users/19d4b11cfa5d4c8ca31463ab1a54d61f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:44.062496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99a133-97a8-49a0-85b2-4d9a2550b837 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:44.081470 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99a133-97a8-49a0-85b2-4d9a2550b837 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:44.151666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3db84-adfa-4c19-8e97-58ac73058211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:44.152317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3db84-adfa-4c19-8e97-58ac73058211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:44.152614 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3db84-adfa-4c19-8e97-58ac73058211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:44.152907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3db84-adfa-4c19-8e97-58ac73058211 admin admin] PATH_INFO: `/v3/projects/7b0d38ca8c254eb4a48521422cfed830` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:44.154714 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3db84-adfa-4c19-8e97-58ac73058211 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b0d38ca8c254eb4a48521422cfed830)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:44.156928 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3db84-adfa-4c19-8e97-58ac73058211 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:44.220541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-3ea3db84-adfa-4c19-8e97-58ac73058211 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b0d38ca8c254eb4a48521422cfed830', 'request_id': 'req-3ea3db84-adfa-4c19-8e97-58ac73058211'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:44.226853 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-3ea3db84-adfa-4c19-8e97-58ac73058211 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b0d38ca8c254eb4a48521422cfed830', 'request_id': 'req-3ea3db84-adfa-4c19-8e97-58ac73058211'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:44.233007 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-3ea3db84-adfa-4c19-8e97-58ac73058211 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b0d38ca8c254eb4a48521422cfed830', 'request_id': 'req-3ea3db84-adfa-4c19-8e97-58ac73058211'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:44.247210 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-3ea3db84-adfa-4c19-8e97-58ac73058211 admin admin] The token cache is being invalidate because project 7b0d38ca8c254eb4a48521422cfed830 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:06:44.247513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-3ea3db84-adfa-4c19-8e97-58ac73058211 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:44.248173 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-3ea3db84-adfa-4c19-8e97-58ac73058211 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:44.249735 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 923/1825] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:06:44 2022] DELETE /identity/v3/projects/7b0d38ca8c254eb4a48521422cfed830 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:44.253977 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb102e1-e5a7-4c99-889d-42a2ee120d2f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:44.255644 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-bdb102e1-e5a7-4c99-889d-42a2ee120d2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:44.269754 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb102e1-e5a7-4c99-889d-42a2ee120d2f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:44.338940 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1350a-c591-49cb-8c65-6a19549d0fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:44.339606 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1350a-c591-49cb-8c65-6a19549d0fb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:44.339888 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1350a-c591-49cb-8c65-6a19549d0fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:44.340169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1350a-c591-49cb-8c65-6a19549d0fb7 admin admin] PATH_INFO: `/v3/users/c684b50e5f184065aaebbe99b27836e9` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:44.341371 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1350a-c591-49cb-8c65-6a19549d0fb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c684b50e5f184065aaebbe99b27836e9)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:44.343342 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1350a-c591-49cb-8c65-6a19549d0fb7 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:44.367748 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:44.398668 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9fb1350a-c591-49cb-8c65-6a19549d0fb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c684b50e5f184065aaebbe99b27836e9'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:44.399346 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9fb1350a-c591-49cb-8c65-6a19549d0fb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c684b50e5f184065aaebbe99b27836e9'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:44.407294 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9fb1350a-c591-49cb-8c65-6a19549d0fb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c684b50e5f184065aaebbe99b27836e9'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:44.420753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9fb1350a-c591-49cb-8c65-6a19549d0fb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c684b50e5f184065aaebbe99b27836e9'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:44.427695 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9fb1350a-c591-49cb-8c65-6a19549d0fb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c684b50e5f184065aaebbe99b27836e9'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:44.433992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 903/1826] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:06:44 2022] DELETE /identity/v3/users/c684b50e5f184065aaebbe99b27836e9 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:44.630590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3db84-adfa-4c19-8e97-58ac73058211 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:44.632826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3ea3db84-adfa-4c19-8e97-58ac73058211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:44.651338 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3db84-adfa-4c19-8e97-58ac73058211 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:44.711640 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:44.744383 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf57b5bd-5917-427b-973d-0343fd182c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:44.745190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57b5bd-5917-427b-973d-0343fd182c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:44.745596 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57b5bd-5917-427b-973d-0343fd182c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:44.745926 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57b5bd-5917-427b-973d-0343fd182c8b admin admin] PATH_INFO: `/v3/projects/82bad4f651b24ea7876d4723a0666eac` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:44.747132 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf57b5bd-5917-427b-973d-0343fd182c8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=82bad4f651b24ea7876d4723a0666eac)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:44.749189 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf57b5bd-5917-427b-973d-0343fd182c8b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:44.829968 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cf57b5bd-5917-427b-973d-0343fd182c8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82bad4f651b24ea7876d4723a0666eac', 'request_id': 'req-cf57b5bd-5917-427b-973d-0343fd182c8b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:44.837107 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cf57b5bd-5917-427b-973d-0343fd182c8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82bad4f651b24ea7876d4723a0666eac', 'request_id': 'req-cf57b5bd-5917-427b-973d-0343fd182c8b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:44.846537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cf57b5bd-5917-427b-973d-0343fd182c8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82bad4f651b24ea7876d4723a0666eac', 'request_id': 'req-cf57b5bd-5917-427b-973d-0343fd182c8b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:44.868027 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cf57b5bd-5917-427b-973d-0343fd182c8b admin admin] The token cache is being invalidate because project 82bad4f651b24ea7876d4723a0666eac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:06:44.870296 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cf57b5bd-5917-427b-973d-0343fd182c8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:44.871115 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cf57b5bd-5917-427b-973d-0343fd182c8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:06:44.873942 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 924/1827] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:06:44 2022] DELETE /identity/v3/projects/82bad4f651b24ea7876d4723a0666eac => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:44.878549 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845ef2a-471d-460b-9467-344dce5aebd0 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:44.879446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845ef2a-471d-460b-9467-344dce5aebd0 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:44.879999 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845ef2a-471d-460b-9467-344dce5aebd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:44.907387 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c845ef2a-471d-460b-9467-344dce5aebd0 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:44.909127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c845ef2a-471d-460b-9467-344dce5aebd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:44.987931 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 904/1828] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:06:44.996885 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9a635-f59b-4c25-87bf-244b6a0dc9d2 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:44.997258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9a635-f59b-4c25-87bf-244b6a0dc9d2 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:44.997597 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9a635-f59b-4c25-87bf-244b6a0dc9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:45.018650 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-49a9a635-f59b-4c25-87bf-244b6a0dc9d2 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:45.020155 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-49a9a635-f59b-4c25-87bf-244b6a0dc9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:45.030750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 925/1829] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:06:45.035339 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845ef2a-471d-460b-9467-344dce5aebd0 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:45.046349 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845ef2a-471d-460b-9467-344dce5aebd0 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:45.051416 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe2b05-31ee-460c-9f8f-2f1dd856333b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:45.052084 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe2b05-31ee-460c-9f8f-2f1dd856333b admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:45.052300 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe2b05-31ee-460c-9f8f-2f1dd856333b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:45.052579 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe2b05-31ee-460c-9f8f-2f1dd856333b admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:45.053286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe2b05-31ee-460c-9f8f-2f1dd856333b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:06:45.053639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe2b05-31ee-460c-9f8f-2f1dd856333b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:45.055454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe2b05-31ee-460c-9f8f-2f1dd856333b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:45.068738 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 905/1830] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:06:45.072068 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9a635-f59b-4c25-87bf-244b6a0dc9d2 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:45.082842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9a635-f59b-4c25-87bf-244b6a0dc9d2 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:45.088525 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994813b-e123-4146-9a8e-8793c1881ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:45.089119 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994813b-e123-4146-9a8e-8793c1881ac2 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:45.089378 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994813b-e123-4146-9a8e-8793c1881ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:45.089882 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994813b-e123-4146-9a8e-8793c1881ac2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:45.090716 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994813b-e123-4146-9a8e-8793c1881ac2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:45.092446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994813b-e123-4146-9a8e-8793c1881ac2 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:45.132273 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 926/1831] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:06:45 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:06:45.133661 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe2b05-31ee-460c-9f8f-2f1dd856333b admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:45.144322 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe2b05-31ee-460c-9f8f-2f1dd856333b admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:45.226131 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d59eb-62d5-4092-8585-371e3460bf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:45.226536 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d59eb-62d5-4092-8585-371e3460bf1c admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:45.226881 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d59eb-62d5-4092-8585-371e3460bf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:45.227049 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d59eb-62d5-4092-8585-371e3460bf1c admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:45.228106 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d59eb-62d5-4092-8585-371e3460bf1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:45.229699 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d59eb-62d5-4092-8585-371e3460bf1c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:45.254100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 906/1832] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:06:45 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:06:45.259067 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994813b-e123-4146-9a8e-8793c1881ac2 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:45.269412 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994813b-e123-4146-9a8e-8793c1881ac2 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:45.274341 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55b3d5-b4a3-4812-86be-0d2de09027d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:45.274957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55b3d5-b4a3-4812-86be-0d2de09027d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:45.275220 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55b3d5-b4a3-4812-86be-0d2de09027d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:45.275489 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55b3d5-b4a3-4812-86be-0d2de09027d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:45.276128 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd55b3d5-b4a3-4812-86be-0d2de09027d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:45.278024 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd55b3d5-b4a3-4812-86be-0d2de09027d0 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:45.289500 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 927/1833] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:06:45.293326 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d59eb-62d5-4092-8585-371e3460bf1c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:45.304807 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d59eb-62d5-4092-8585-371e3460bf1c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:45.311040 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d753d6-269d-4a1c-b52e-2119bb8692f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:45.311713 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d753d6-269d-4a1c-b52e-2119bb8692f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:45.311996 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d753d6-269d-4a1c-b52e-2119bb8692f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:45.312265 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d753d6-269d-4a1c-b52e-2119bb8692f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:45.312945 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d753d6-269d-4a1c-b52e-2119bb8692f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:45.314557 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d753d6-269d-4a1c-b52e-2119bb8692f7 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:45.326508 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 907/1834] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:06:45.330864 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55b3d5-b4a3-4812-86be-0d2de09027d0 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:45.341726 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55b3d5-b4a3-4812-86be-0d2de09027d0 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:45.346497 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e5be0-4bdc-49fa-a27b-6089160cd350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:45.347071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e5be0-4bdc-49fa-a27b-6089160cd350 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:45.347345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e5be0-4bdc-49fa-a27b-6089160cd350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:45.347610 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e5be0-4bdc-49fa-a27b-6089160cd350 admin admin] PATH_INFO: `/v3/projects/97ccd6d348ab4a0da96fdc0b8ae0d230/users/272d2775336246cb8a1d04efa3a585ad/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:45.364226 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e5be0-4bdc-49fa-a27b-6089160cd350 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97ccd6d348ab4a0da96fdc0b8ae0d230, user_id=272d2775336246cb8a1d04efa3a585ad, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:45.367472 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e5be0-4bdc-49fa-a27b-6089160cd350 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:45.381629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 928/1835] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:06:45 2022] PUT /identity/v3/projects/97ccd6d348ab4a0da96fdc0b8ae0d230/users/272d2775336246cb8a1d04efa3a585ad/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:45.392187 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521b21f-5f9c-49c8-bb66-a790e6862237 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:45.392378 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521b21f-5f9c-49c8-bb66-a790e6862237 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:45.392620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521b21f-5f9c-49c8-bb66-a790e6862237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:45.400918 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:45.425944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c521b21f-5f9c-49c8-bb66-a790e6862237 None None] MFA Rules not processed for user `272d2775336246cb8a1d04efa3a585ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:45.427173 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c521b21f-5f9c-49c8-bb66-a790e6862237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:45.498383 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 908/1836] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:06:45.503369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e5be0-4bdc-49fa-a27b-6089160cd350 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:45.604572 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e5be0-4bdc-49fa-a27b-6089160cd350 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:45.609411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99a9c5-6953-4dc5-9956-91a9570aaf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:45.610030 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99a9c5-6953-4dc5-9956-91a9570aaf65 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:45.610313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99a9c5-6953-4dc5-9956-91a9570aaf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:45.611114 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99a9c5-6953-4dc5-9956-91a9570aaf65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:45.611224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99a9c5-6953-4dc5-9956-91a9570aaf65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:45.613349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99a9c5-6953-4dc5-9956-91a9570aaf65 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:45.620449 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:45.649317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 929/1837] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:06:45 2022] POST /identity/v3/projects => generated 375 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:06:45.653233 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521b21f-5f9c-49c8-bb66-a790e6862237 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:45.665340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521b21f-5f9c-49c8-bb66-a790e6862237 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:45.739852 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd33e2f-9664-4aef-ad06-2c58c774eaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:45.740571 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd33e2f-9664-4aef-ad06-2c58c774eaa2 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:45.740885 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd33e2f-9664-4aef-ad06-2c58c774eaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:45.741153 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd33e2f-9664-4aef-ad06-2c58c774eaa2 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:45.741773 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd33e2f-9664-4aef-ad06-2c58c774eaa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:45.743676 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd33e2f-9664-4aef-ad06-2c58c774eaa2 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:45.766397 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 909/1838] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:06:45 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:06:45.770205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99a9c5-6953-4dc5-9956-91a9570aaf65 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:45.782748 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99a9c5-6953-4dc5-9956-91a9570aaf65 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:45.788498 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43349614-08c6-481d-9f83-63064f836a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:45.789241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43349614-08c6-481d-9f83-63064f836a37 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:45.789533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43349614-08c6-481d-9f83-63064f836a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:45.789806 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43349614-08c6-481d-9f83-63064f836a37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:45.790459 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43349614-08c6-481d-9f83-63064f836a37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:45.792105 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43349614-08c6-481d-9f83-63064f836a37 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:45.808283 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 930/1839] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:06:45.809612 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd33e2f-9664-4aef-ad06-2c58c774eaa2 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:45.822055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd33e2f-9664-4aef-ad06-2c58c774eaa2 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:45.827322 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d6589-156e-4299-8f7d-de2188d673d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:45.828051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d6589-156e-4299-8f7d-de2188d673d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:45.828336 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d6589-156e-4299-8f7d-de2188d673d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:45.828628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d6589-156e-4299-8f7d-de2188d673d6 admin admin] PATH_INFO: `/v3/projects/287cb4cba8234b85b13f014a8bf24986/users/722bf580b59c4a41abebec1acc0e0763/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:45.846135 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d6589-156e-4299-8f7d-de2188d673d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=287cb4cba8234b85b13f014a8bf24986, user_id=722bf580b59c4a41abebec1acc0e0763, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:45.849347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d6589-156e-4299-8f7d-de2188d673d6 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:45.867165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 910/1840] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:06:45 2022] PUT /identity/v3/projects/287cb4cba8234b85b13f014a8bf24986/users/722bf580b59c4a41abebec1acc0e0763/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:45.871775 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43349614-08c6-481d-9f83-63064f836a37 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:45.882453 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43349614-08c6-481d-9f83-63064f836a37 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:45.940140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ddb718-114a-4f08-82a6-4758f74ce6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:45.940647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ddb718-114a-4f08-82a6-4758f74ce6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:45.940956 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ddb718-114a-4f08-82a6-4758f74ce6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:45.941225 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ddb718-114a-4f08-82a6-4758f74ce6b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:45.941848 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ddb718-114a-4f08-82a6-4758f74ce6b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:45.944064 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ddb718-114a-4f08-82a6-4758f74ce6b9 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:45.953931 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 931/1841] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:06:45.958001 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d6589-156e-4299-8f7d-de2188d673d6 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:45.968730 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d6589-156e-4299-8f7d-de2188d673d6 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:45.974341 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac637ab-f9a1-4646-ae6d-ab017a754aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:45.975927 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac637ab-f9a1-4646-ae6d-ab017a754aa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:45.976212 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac637ab-f9a1-4646-ae6d-ab017a754aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:45.976491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac637ab-f9a1-4646-ae6d-ab017a754aa4 admin admin] PATH_INFO: `/v3/projects/287cb4cba8234b85b13f014a8bf24986/users/722bf580b59c4a41abebec1acc0e0763/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:45.978514 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac637ab-f9a1-4646-ae6d-ab017a754aa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=287cb4cba8234b85b13f014a8bf24986, user_id=722bf580b59c4a41abebec1acc0e0763, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:45.981209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac637ab-f9a1-4646-ae6d-ab017a754aa4 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:45.993336 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 911/1842] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:06:45 2022] PUT /identity/v3/projects/287cb4cba8234b85b13f014a8bf24986/users/722bf580b59c4a41abebec1acc0e0763/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:06:46.004971 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9da1a-89f0-4097-b50b-35de79a2a1ef None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:46.005256 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9da1a-89f0-4097-b50b-35de79a2a1ef None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:46.005751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9da1a-89f0-4097-b50b-35de79a2a1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:46.038117 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a3a9da1a-89f0-4097-b50b-35de79a2a1ef None None] MFA Rules not processed for user `722bf580b59c4a41abebec1acc0e0763`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:06:46.039343 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a3a9da1a-89f0-4097-b50b-35de79a2a1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:46.108967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 932/1843] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:06:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:06:46.120220 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac637ab-f9a1-4646-ae6d-ab017a754aa4 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:06:46.121946 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-dac637ab-f9a1-4646-ae6d-ab017a754aa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:06:46.182923 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:46.191659 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac637ab-f9a1-4646-ae6d-ab017a754aa4 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:06:46.204000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd78d1-b159-4692-aea4-42e6f1ed906c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:06:46.204848 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd78d1-b159-4692-aea4-42e6f1ed906c service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:06:46.205290 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd78d1-b159-4692-aea4-42e6f1ed906c service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:06:46.205712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd78d1-b159-4692-aea4-42e6f1ed906c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:06:46.350253 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd78d1-b159-4692-aea4-42e6f1ed906c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:06:46.351979 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd78d1-b159-4692-aea4-42e6f1ed906c service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:06:46.361733 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 912/1844] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:06:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:06:56.106127 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=73,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:06:56.357385 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=27 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:07:44.055649 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9da1a-89f0-4097-b50b-35de79a2a1ef None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:07:44.057317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a3a9da1a-89f0-4097-b50b-35de79a2a1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:07:44.125245 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9da1a-89f0-4097-b50b-35de79a2a1ef None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:07:44.130192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80362346-3a94-4bb6-bebd-03cd6743298d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfbaaf32475a4c82a34948602d8f4e35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:07:44.130792 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80362346-3a94-4bb6-bebd-03cd6743298d service placement] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:07:44.131076 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80362346-3a94-4bb6-bebd-03cd6743298d service placement] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:07:44.131351 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80362346-3a94-4bb6-bebd-03cd6743298d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:07:44.133316 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-80362346-3a94-4bb6-bebd-03cd6743298d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:07:44.144943 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80362346-3a94-4bb6-bebd-03cd6743298d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:07:44.146516 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80362346-3a94-4bb6-bebd-03cd6743298d service placement] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:07:44.156326 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 933/1845] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:07:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:07:45.343932 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff8900-548b-49d8-b330-76d856a0ff28 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:07:45.344452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff8900-548b-49d8-b330-76d856a0ff28 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:07:45.344452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff8900-548b-49d8-b330-76d856a0ff28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:07:45.364223 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-d2ff8900-548b-49d8-b330-76d856a0ff28 None None] MFA Rules not processed for user `7a59f547a20e48ee88e3e620ba1e87f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:07:45.365463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d2ff8900-548b-49d8-b330-76d856a0ff28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:07:45.422933 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 913/1846] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Oct 20 06:07:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:07:45.465408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80362346-3a94-4bb6-bebd-03cd6743298d service placement] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:07:45.466812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-80362346-3a94-4bb6-bebd-03cd6743298d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:07:45.544160 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80362346-3a94-4bb6-bebd-03cd6743298d service placement] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:07:45.549615 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40bf4c8-4736-4b3b-8cb8-866ddcbc6eb6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeeab506ab5c4d6386fc899086f0abf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:07:45.550502 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bf4c8-4736-4b3b-8cb8-866ddcbc6eb6 service swift] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:07:45.550784 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bf4c8-4736-4b3b-8cb8-866ddcbc6eb6 service swift] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:07:45.551061 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bf4c8-4736-4b3b-8cb8-866ddcbc6eb6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:07:45.559930 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40bf4c8-4736-4b3b-8cb8-866ddcbc6eb6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:07:45.561550 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40bf4c8-4736-4b3b-8cb8-866ddcbc6eb6 service swift] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:07:45.572490 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 934/1847] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Oct 20 06:07:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 06:07:55.416351 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=25 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:07:55.566945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=64 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:07:59.902315 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad172dd-05bc-40a2-9928-cc6e3e888ac8 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:07:59.902860 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad172dd-05bc-40a2-9928-cc6e3e888ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:07:59.902860 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad172dd-05bc-40a2-9928-cc6e3e888ac8 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:07:59.904184 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 914/1848] 10.222.0.20 () {54 vars in 921 bytes} [Thu Oct 20 06:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 06:07:59.915077 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d65fb5-e8b1-44e1-bd3c-91309e68d773 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:07:59.915605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d65fb5-e8b1-44e1-bd3c-91309e68d773 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:07:59.915605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d65fb5-e8b1-44e1-bd3c-91309e68d773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:07:59.935667 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-99d65fb5-e8b1-44e1-bd3c-91309e68d773 None None] MFA Rules not processed for user `ae2c1c886cd94d9f90082f66e7e8c9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:07:59.937219 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-99d65fb5-e8b1-44e1-bd3c-91309e68d773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:08:00.022475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 935/1849] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Oct 20 06:07:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:08:00.035504 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad172dd-05bc-40a2-9928-cc6e3e888ac8 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:08:00.037776 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-cad172dd-05bc-40a2-9928-cc6e3e888ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:08:00.144185 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad172dd-05bc-40a2-9928-cc6e3e888ac8 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:08:00.148048 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4b726-08d9-476a-a366-e88327e241a2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dcfe5244bd04c00b6dde1d6f795d4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:08:00.149493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4b726-08d9-476a-a366-e88327e241a2 service glance] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:08:00.149926 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4b726-08d9-476a-a366-e88327e241a2 service glance] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:08:00.150207 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4b726-08d9-476a-a366-e88327e241a2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:08:00.162299 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4b726-08d9-476a-a366-e88327e241a2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:08:00.163947 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4b726-08d9-476a-a366-e88327e241a2 service glance] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:08:00.177251 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 915/1850] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:08:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:08:00.255083 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378b1a5-7d78-41fb-bd4f-4fb2b7cc4af6 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:08:00.255347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378b1a5-7d78-41fb-bd4f-4fb2b7cc4af6 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:08:00.255621 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378b1a5-7d78-41fb-bd4f-4fb2b7cc4af6 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:08:00.257136 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 936/1851] 10.222.0.20 () {54 vars in 921 bytes} [Thu Oct 20 06:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 06:08:00.262689 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fed404-face-476e-87d4-99ea762202de None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:08:00.263140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fed404-face-476e-87d4-99ea762202de None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:08:00.263417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fed404-face-476e-87d4-99ea762202de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:08:00.285215 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-15fed404-face-476e-87d4-99ea762202de None None] MFA Rules not processed for user `9072ce66e7174837aeb8c7898026fcce`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:08:00.286411 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-15fed404-face-476e-87d4-99ea762202de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:08:00.342232 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 916/1852] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Oct 20 06:08:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:08:00.355020 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378b1a5-7d78-41fb-bd4f-4fb2b7cc4af6 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:08:00.356767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e378b1a5-7d78-41fb-bd4f-4fb2b7cc4af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:08:00.370001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378b1a5-7d78-41fb-bd4f-4fb2b7cc4af6 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:08:00.373559 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1b742-0afa-4aee-ba26-c53d08fec3a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:08:00.374287 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1b742-0afa-4aee-ba26-c53d08fec3a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:08:00.374651 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1b742-0afa-4aee-ba26-c53d08fec3a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:08:00.374988 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1b742-0afa-4aee-ba26-c53d08fec3a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:08:00.382849 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1b742-0afa-4aee-ba26-c53d08fec3a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:08:00.384371 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1b742-0afa-4aee-ba26-c53d08fec3a6 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:08:00.392377 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 937/1853] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:08:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:08:01.524473 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63a86d-f6b1-4527-a25c-1f397680272a None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:08:01.524983 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63a86d-f6b1-4527-a25c-1f397680272a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:08:01.525110 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63a86d-f6b1-4527-a25c-1f397680272a None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:08:01.526410 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 917/1854] 10.222.0.20 () {54 vars in 921 bytes} [Thu Oct 20 06:08:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 06:08:01.531602 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2be43d-4611-4b35-98e3-56609578a772 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:08:01.532025 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2be43d-4611-4b35-98e3-56609578a772 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:08:01.532141 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2be43d-4611-4b35-98e3-56609578a772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:08:01.551457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-5e2be43d-4611-4b35-98e3-56609578a772 None None] MFA Rules not processed for user `ae2c1c886cd94d9f90082f66e7e8c9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:08:01.552645 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5e2be43d-4611-4b35-98e3-56609578a772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:08:01.562077 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 938/1855] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Oct 20 06:08:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:08:01.573170 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63a86d-f6b1-4527-a25c-1f397680272a None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:08:01.574824 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6f63a86d-f6b1-4527-a25c-1f397680272a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:08:01.588424 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63a86d-f6b1-4527-a25c-1f397680272a None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:08:01.591922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b09693a-18fa-4f56-ab81-deb976975f65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:08:01.592499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09693a-18fa-4f56-ab81-deb976975f65 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:08:01.592806 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09693a-18fa-4f56-ab81-deb976975f65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:08:01.593074 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09693a-18fa-4f56-ab81-deb976975f65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:08:01.603035 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b09693a-18fa-4f56-ab81-deb976975f65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:08:01.604507 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b09693a-18fa-4f56-ab81-deb976975f65 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:08:01.614814 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 918/1856] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:08:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:08:11.546609 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=40 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:08:11.610389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=64 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:09:29.838699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2be43d-4611-4b35-98e3-56609578a772 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:09:29.847701 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2be43d-4611-4b35-98e3-56609578a772 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:09:29.851180 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c965737-e13c-4001-a6a1-7709f6deac27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:09:29.851767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c965737-e13c-4001-a6a1-7709f6deac27 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:09:29.852039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c965737-e13c-4001-a6a1-7709f6deac27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:09:29.852305 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c965737-e13c-4001-a6a1-7709f6deac27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:09:29.854662 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3c965737-e13c-4001-a6a1-7709f6deac27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:09:29.868226 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c965737-e13c-4001-a6a1-7709f6deac27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:09:29.869680 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c965737-e13c-4001-a6a1-7709f6deac27 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:09:29.880194 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 939/1857] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:09:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:09:32.117406 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b09693a-18fa-4f56-ab81-deb976975f65 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:09:32.119273 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2b09693a-18fa-4f56-ab81-deb976975f65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:09:32.134677 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b09693a-18fa-4f56-ab81-deb976975f65 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:09:32.140486 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc90358-2eb9-4860-8a76-72e5e6397a89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:09:32.140908 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc90358-2eb9-4860-8a76-72e5e6397a89 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:09:32.141194 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc90358-2eb9-4860-8a76-72e5e6397a89 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:09:32.141467 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc90358-2eb9-4860-8a76-72e5e6397a89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:09:32.143167 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2dc90358-2eb9-4860-8a76-72e5e6397a89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:09:32.202349 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc90358-2eb9-4860-8a76-72e5e6397a89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:09:32.203936 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc90358-2eb9-4860-8a76-72e5e6397a89 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:09:32.213233 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 919/1858] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:09:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:09:39.204238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c965737-e13c-4001-a6a1-7709f6deac27 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:09:39.214455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c965737-e13c-4001-a6a1-7709f6deac27 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:09:39.218027 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba2a8f-78b1-44ac-bce8-2a19bf90de96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:09:39.218619 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba2a8f-78b1-44ac-bce8-2a19bf90de96 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:09:39.218896 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba2a8f-78b1-44ac-bce8-2a19bf90de96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:09:39.219165 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba2a8f-78b1-44ac-bce8-2a19bf90de96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:09:39.220917 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-23ba2a8f-78b1-44ac-bce8-2a19bf90de96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:09:39.281636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba2a8f-78b1-44ac-bce8-2a19bf90de96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:09:39.283089 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba2a8f-78b1-44ac-bce8-2a19bf90de96 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:09:39.292200 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 940/1859] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:09:39 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:09:42.208833 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=28 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:09:42.864100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc90358-2eb9-4860-8a76-72e5e6397a89 service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:09:42.865515 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2dc90358-2eb9-4860-8a76-72e5e6397a89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:09:42.880323 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc90358-2eb9-4860-8a76-72e5e6397a89 service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:09:42.886302 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3953c51-eec1-4daf-aa25-4978175398e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:09:42.886926 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3953c51-eec1-4daf-aa25-4978175398e7 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:09:42.887204 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3953c51-eec1-4daf-aa25-4978175398e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:09:42.887478 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3953c51-eec1-4daf-aa25-4978175398e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:09:42.889151 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b3953c51-eec1-4daf-aa25-4978175398e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:09:42.946676 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3953c51-eec1-4daf-aa25-4978175398e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:09:42.948255 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3953c51-eec1-4daf-aa25-4978175398e7 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:09:42.957194 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 920/1860] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:09:42 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:09:46.501796 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba2a8f-78b1-44ac-bce8-2a19bf90de96 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:09:46.510753 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba2a8f-78b1-44ac-bce8-2a19bf90de96 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:09:46.514364 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81b3b7-47c3-4215-ae8b-3557c95acba7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:09:46.514965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81b3b7-47c3-4215-ae8b-3557c95acba7 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:09:46.515242 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81b3b7-47c3-4215-ae8b-3557c95acba7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:09:46.515512 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81b3b7-47c3-4215-ae8b-3557c95acba7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:09:46.517261 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-de81b3b7-47c3-4215-ae8b-3557c95acba7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:09:46.576674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81b3b7-47c3-4215-ae8b-3557c95acba7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:09:46.578064 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81b3b7-47c3-4215-ae8b-3557c95acba7 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:09:46.587439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 941/1861] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:09:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:09:49.133697 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3953c51-eec1-4daf-aa25-4978175398e7 service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:09:49.142871 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3953c51-eec1-4daf-aa25-4978175398e7 service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:09:49.146458 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d5abc-cd71-4358-9bf1-0f0f2f59433d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:09:49.147055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d5abc-cd71-4358-9bf1-0f0f2f59433d service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:09:49.147334 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d5abc-cd71-4358-9bf1-0f0f2f59433d service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:09:49.147609 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d5abc-cd71-4358-9bf1-0f0f2f59433d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:09:49.149394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1c8d5abc-cd71-4358-9bf1-0f0f2f59433d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:09:49.162457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d5abc-cd71-4358-9bf1-0f0f2f59433d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:09:49.163898 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d5abc-cd71-4358-9bf1-0f0f2f59433d service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:09:49.174672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 921/1862] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:09:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:09:56.583570 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=62 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:09:59.169912 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=34 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:10:37.513843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81b3b7-47c3-4215-ae8b-3557c95acba7 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:10:37.522345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81b3b7-47c3-4215-ae8b-3557c95acba7 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:10:37.526030 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663afd28-feeb-4c93-9bed-bbe574aff161 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:10:37.526630 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663afd28-feeb-4c93-9bed-bbe574aff161 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:10:37.526905 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663afd28-feeb-4c93-9bed-bbe574aff161 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:10:37.527386 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663afd28-feeb-4c93-9bed-bbe574aff161 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:10:37.528921 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-663afd28-feeb-4c93-9bed-bbe574aff161 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:10:37.600522 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663afd28-feeb-4c93-9bed-bbe574aff161 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:10:37.602118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663afd28-feeb-4c93-9bed-bbe574aff161 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:10:37.610830 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 942/1863] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:10:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:10:38.154535 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d5abc-cd71-4358-9bf1-0f0f2f59433d service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:10:38.163046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d5abc-cd71-4358-9bf1-0f0f2f59433d service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:10:38.167353 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a4792-4960-4f52-af92-3c964f1f900f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:10:38.167951 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a4792-4960-4f52-af92-3c964f1f900f service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:10:38.168232 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a4792-4960-4f52-af92-3c964f1f900f service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:10:38.168504 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a4792-4960-4f52-af92-3c964f1f900f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:10:38.170247 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-db7a4792-4960-4f52-af92-3c964f1f900f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:10:38.180449 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a4792-4960-4f52-af92-3c964f1f900f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:10:38.181997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a4792-4960-4f52-af92-3c964f1f900f service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:10:38.189788 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 922/1864] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:10:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:10:47.606883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=28 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:10:48.186306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=6 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:11:20.876050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663afd28-feeb-4c93-9bed-bbe574aff161 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:11:20.885935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663afd28-feeb-4c93-9bed-bbe574aff161 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:11:20.889845 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa27b19-f937-43c4-84de-91be9954f72e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:11:20.890573 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa27b19-f937-43c4-84de-91be9954f72e service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:11:20.892548 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa27b19-f937-43c4-84de-91be9954f72e service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:11:20.892548 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa27b19-f937-43c4-84de-91be9954f72e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:11:20.893010 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-efa27b19-f937-43c4-84de-91be9954f72e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:11:20.953430 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa27b19-f937-43c4-84de-91be9954f72e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:11:20.954911 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa27b19-f937-43c4-84de-91be9954f72e service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:11:20.964540 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 943/1865] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:11:20 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:11:30.960499 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=28 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:11:46.334742 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a4792-4960-4f52-af92-3c964f1f900f service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:11:46.344031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a4792-4960-4f52-af92-3c964f1f900f service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:11:46.347975 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bf2d1-ea28-4e4f-afc4-1968a9497f71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:11:46.348572 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bf2d1-ea28-4e4f-afc4-1968a9497f71 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:11:46.348894 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bf2d1-ea28-4e4f-afc4-1968a9497f71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:11:46.349167 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bf2d1-ea28-4e4f-afc4-1968a9497f71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:11:46.357528 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191bf2d1-ea28-4e4f-afc4-1968a9497f71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:11:46.358940 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191bf2d1-ea28-4e4f-afc4-1968a9497f71 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:11:46.368402 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 923/1866] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:11:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:11:56.363732 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=6 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:12:46.745675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa27b19-f937-43c4-84de-91be9954f72e service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:12:46.864522 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa27b19-f937-43c4-84de-91be9954f72e service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:12:46.869509 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ae8c0-947e-4a1a-a669-db39dff7a4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:12:46.870130 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ae8c0-947e-4a1a-a669-db39dff7a4eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:12:46.870402 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ae8c0-947e-4a1a-a669-db39dff7a4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:12:46.870668 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ae8c0-947e-4a1a-a669-db39dff7a4eb admin admin] PATH_INFO: `/v3/users/272d2775336246cb8a1d04efa3a585ad` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:12:46.871783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ae8c0-947e-4a1a-a669-db39dff7a4eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=272d2775336246cb8a1d04efa3a585ad)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:12:46.873705 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ae8c0-947e-4a1a-a669-db39dff7a4eb admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:12:46.918701 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fc6ae8c0-947e-4a1a-a669-db39dff7a4eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '272d2775336246cb8a1d04efa3a585ad'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:12:46.919335 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fc6ae8c0-947e-4a1a-a669-db39dff7a4eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '272d2775336246cb8a1d04efa3a585ad'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:12:46.926581 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fc6ae8c0-947e-4a1a-a669-db39dff7a4eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '272d2775336246cb8a1d04efa3a585ad'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:12:46.938976 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fc6ae8c0-947e-4a1a-a669-db39dff7a4eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '272d2775336246cb8a1d04efa3a585ad'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:12:46.939599 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fc6ae8c0-947e-4a1a-a669-db39dff7a4eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '272d2775336246cb8a1d04efa3a585ad'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:12:46.951447 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 944/1867] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:12:46 2022] DELETE /identity/v3/users/272d2775336246cb8a1d04efa3a585ad => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:12:46.955894 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d8ec7-554a-4335-96ea-a7d35605e87b None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:12:46.956364 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d8ec7-554a-4335-96ea-a7d35605e87b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:12:46.956459 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d8ec7-554a-4335-96ea-a7d35605e87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:12:47.005757 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-cc4d8ec7-554a-4335-96ea-a7d35605e87b None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:12:47.007232 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-cc4d8ec7-554a-4335-96ea-a7d35605e87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:12:47.067580 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 924/1868] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:12:47.078025 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ae8c0-947e-4a1a-a669-db39dff7a4eb admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:12:47.079398 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-fc6ae8c0-947e-4a1a-a669-db39dff7a4eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:12:47.121932 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:12:47.155889 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ae8c0-947e-4a1a-a669-db39dff7a4eb admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:12:47.159747 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d194e3-b680-4449-9503-d169c38b2d5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:12:47.160347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d194e3-b680-4449-9503-d169c38b2d5e service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:12:47.160635 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d194e3-b680-4449-9503-d169c38b2d5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:12:47.160931 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d194e3-b680-4449-9503-d169c38b2d5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:12:47.171320 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d194e3-b680-4449-9503-d169c38b2d5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:12:47.172765 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d194e3-b680-4449-9503-d169c38b2d5e service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:12:47.183817 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 945/1869] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:12:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:12:47.515983 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4d8ec7-554a-4335-96ea-a7d35605e87b None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:12:47.517533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-cc4d8ec7-554a-4335-96ea-a7d35605e87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:12:47.529602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4d8ec7-554a-4335-96ea-a7d35605e87b None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:12:47.534363 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6a888-c5eb-47e5-af30-65fe30d546ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:12:47.535003 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6a888-c5eb-47e5-af30-65fe30d546ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:12:47.535285 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6a888-c5eb-47e5-af30-65fe30d546ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:12:47.535561 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6a888-c5eb-47e5-af30-65fe30d546ed admin admin] PATH_INFO: `/v3/projects/97ccd6d348ab4a0da96fdc0b8ae0d230` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:12:47.536822 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a6a888-c5eb-47e5-af30-65fe30d546ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=97ccd6d348ab4a0da96fdc0b8ae0d230)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:12:47.538937 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a6a888-c5eb-47e5-af30-65fe30d546ed admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:12:47.598477 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d6a6a888-c5eb-47e5-af30-65fe30d546ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97ccd6d348ab4a0da96fdc0b8ae0d230', 'request_id': 'req-d6a6a888-c5eb-47e5-af30-65fe30d546ed'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:12:47.604108 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d6a6a888-c5eb-47e5-af30-65fe30d546ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97ccd6d348ab4a0da96fdc0b8ae0d230', 'request_id': 'req-d6a6a888-c5eb-47e5-af30-65fe30d546ed'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:12:47.610347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d6a6a888-c5eb-47e5-af30-65fe30d546ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97ccd6d348ab4a0da96fdc0b8ae0d230', 'request_id': 'req-d6a6a888-c5eb-47e5-af30-65fe30d546ed'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:12:47.622438 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d6a6a888-c5eb-47e5-af30-65fe30d546ed admin admin] The token cache is being invalidate because project 97ccd6d348ab4a0da96fdc0b8ae0d230 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:12:47.622720 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d6a6a888-c5eb-47e5-af30-65fe30d546ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:12:47.623334 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d6a6a888-c5eb-47e5-af30-65fe30d546ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:12:47.624827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 925/1870] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:12:47 2022] DELETE /identity/v3/projects/97ccd6d348ab4a0da96fdc0b8ae0d230 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:12:47.629154 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d194e3-b680-4449-9503-d169c38b2d5e service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:12:47.639515 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d194e3-b680-4449-9503-d169c38b2d5e service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:12:47.695906 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a5be9-d105-47eb-8ff4-c932d4abe35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:12:47.696549 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a5be9-d105-47eb-8ff4-c932d4abe35f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:12:47.696851 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a5be9-d105-47eb-8ff4-c932d4abe35f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:12:47.697119 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a5be9-d105-47eb-8ff4-c932d4abe35f admin admin] PATH_INFO: `/v3/users/722bf580b59c4a41abebec1acc0e0763` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:12:47.698237 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a5be9-d105-47eb-8ff4-c932d4abe35f admin admin] RBAC: Authorizing `identity:delete_user(user_id=722bf580b59c4a41abebec1acc0e0763)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:12:47.700135 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a5be9-d105-47eb-8ff4-c932d4abe35f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:12:47.746571 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c32a5be9-d105-47eb-8ff4-c932d4abe35f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '722bf580b59c4a41abebec1acc0e0763'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:12:47.747201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c32a5be9-d105-47eb-8ff4-c932d4abe35f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '722bf580b59c4a41abebec1acc0e0763'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:12:47.756312 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c32a5be9-d105-47eb-8ff4-c932d4abe35f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '722bf580b59c4a41abebec1acc0e0763'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:12:47.769176 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c32a5be9-d105-47eb-8ff4-c932d4abe35f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '722bf580b59c4a41abebec1acc0e0763'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:12:47.769956 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c32a5be9-d105-47eb-8ff4-c932d4abe35f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '722bf580b59c4a41abebec1acc0e0763'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:12:47.782779 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 946/1871] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:12:47 2022] DELETE /identity/v3/users/722bf580b59c4a41abebec1acc0e0763 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:12:48.052472 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6a888-c5eb-47e5-af30-65fe30d546ed admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:12:48.054165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d6a6a888-c5eb-47e5-af30-65fe30d546ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:12:48.067587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6a888-c5eb-47e5-af30-65fe30d546ed admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:12:48.121480 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:12:48.128099 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d0732c-8dd2-46f3-8dab-604e3a2209ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:12:48.128762 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0732c-8dd2-46f3-8dab-604e3a2209ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:12:48.129443 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0732c-8dd2-46f3-8dab-604e3a2209ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:12:48.129722 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0732c-8dd2-46f3-8dab-604e3a2209ed admin admin] PATH_INFO: `/v3/projects/287cb4cba8234b85b13f014a8bf24986` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:12:48.131359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d0732c-8dd2-46f3-8dab-604e3a2209ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=287cb4cba8234b85b13f014a8bf24986)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:12:48.133857 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d0732c-8dd2-46f3-8dab-604e3a2209ed admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:12:48.180011 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b6d0732c-8dd2-46f3-8dab-604e3a2209ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '287cb4cba8234b85b13f014a8bf24986', 'request_id': 'req-b6d0732c-8dd2-46f3-8dab-604e3a2209ed'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:12:48.185656 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b6d0732c-8dd2-46f3-8dab-604e3a2209ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '287cb4cba8234b85b13f014a8bf24986', 'request_id': 'req-b6d0732c-8dd2-46f3-8dab-604e3a2209ed'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:12:48.192031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b6d0732c-8dd2-46f3-8dab-604e3a2209ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '287cb4cba8234b85b13f014a8bf24986', 'request_id': 'req-b6d0732c-8dd2-46f3-8dab-604e3a2209ed'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:12:48.204658 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b6d0732c-8dd2-46f3-8dab-604e3a2209ed admin admin] The token cache is being invalidate because project 287cb4cba8234b85b13f014a8bf24986 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:12:48.204982 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b6d0732c-8dd2-46f3-8dab-604e3a2209ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:12:48.205590 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b6d0732c-8dd2-46f3-8dab-604e3a2209ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:12:48.207078 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 926/1872] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:12:48 2022] DELETE /identity/v3/projects/287cb4cba8234b85b13f014a8bf24986 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:12:48.213683 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed82f6-2e0b-4686-84ad-e2a376043747 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:12:48.213969 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed82f6-2e0b-4686-84ad-e2a376043747 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:12:48.214248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed82f6-2e0b-4686-84ad-e2a376043747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:12:48.265466 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-24ed82f6-2e0b-4686-84ad-e2a376043747 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:12:48.266661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-24ed82f6-2e0b-4686-84ad-e2a376043747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:12:48.279049 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:12:48.331043 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 947/1873] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:12:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:12:48.396844 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224df1d7-ab63-459e-a524-bff1ec9485e6 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:12:48.397188 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224df1d7-ab63-459e-a524-bff1ec9485e6 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:12:48.397467 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224df1d7-ab63-459e-a524-bff1ec9485e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:12:48.416894 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-224df1d7-ab63-459e-a524-bff1ec9485e6 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:12:48.418100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-224df1d7-ab63-459e-a524-bff1ec9485e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:12:48.425952 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 927/1874] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:12:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:12:48.430610 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed82f6-2e0b-4686-84ad-e2a376043747 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:12:48.440180 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed82f6-2e0b-4686-84ad-e2a376043747 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:12:48.445357 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb525ea9-bf61-4990-81ec-06530671f1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:12:48.446078 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb525ea9-bf61-4990-81ec-06530671f1c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:12:48.446326 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb525ea9-bf61-4990-81ec-06530671f1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:12:48.446600 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb525ea9-bf61-4990-81ec-06530671f1c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:12:48.447237 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb525ea9-bf61-4990-81ec-06530671f1c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:12:48.447533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb525ea9-bf61-4990-81ec-06530671f1c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:12:48.449152 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb525ea9-bf61-4990-81ec-06530671f1c7 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:12:48.464537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 948/1875] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:12:48.465277 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224df1d7-ab63-459e-a524-bff1ec9485e6 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:12:48.474555 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224df1d7-ab63-459e-a524-bff1ec9485e6 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:12:48.479433 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff571f-b8fe-4e00-a7a6-098c1edb06b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:12:48.480024 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff571f-b8fe-4e00-a7a6-098c1edb06b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:12:48.480299 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff571f-b8fe-4e00-a7a6-098c1edb06b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:12:48.480566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff571f-b8fe-4e00-a7a6-098c1edb06b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:12:48.481226 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff571f-b8fe-4e00-a7a6-098c1edb06b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:12:48.483123 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff571f-b8fe-4e00-a7a6-098c1edb06b4 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:12:48.516499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 928/1876] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:12:48 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:12:48.517744 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb525ea9-bf61-4990-81ec-06530671f1c7 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:12:48.527569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb525ea9-bf61-4990-81ec-06530671f1c7 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:12:48.582835 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc3404-9116-4d0d-ab24-d8f48599b4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:12:48.583462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc3404-9116-4d0d-ab24-d8f48599b4d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:12:48.583769 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc3404-9116-4d0d-ab24-d8f48599b4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:12:48.584062 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc3404-9116-4d0d-ab24-d8f48599b4d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:12:48.584715 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc3404-9116-4d0d-ab24-d8f48599b4d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:12:48.586601 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc3404-9116-4d0d-ab24-d8f48599b4d9 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:12:48.610833 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 949/1877] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:12:48 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:12:48.614601 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff571f-b8fe-4e00-a7a6-098c1edb06b4 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:12:48.625335 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff571f-b8fe-4e00-a7a6-098c1edb06b4 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:12:48.630232 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d6af3-4929-43f1-9088-91e4da8889a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:12:48.630953 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d6af3-4929-43f1-9088-91e4da8889a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:12:48.631242 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d6af3-4929-43f1-9088-91e4da8889a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:12:48.631511 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d6af3-4929-43f1-9088-91e4da8889a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:12:48.632138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d6af3-4929-43f1-9088-91e4da8889a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:12:48.633749 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d6af3-4929-43f1-9088-91e4da8889a0 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:12:48.644092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 929/1878] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:12:48.648243 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc3404-9116-4d0d-ab24-d8f48599b4d9 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:12:48.658763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc3404-9116-4d0d-ab24-d8f48599b4d9 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:12:48.664816 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38151f9-a5c4-4151-9063-17c9e8974daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:12:48.665421 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38151f9-a5c4-4151-9063-17c9e8974daa admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:12:48.665702 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38151f9-a5c4-4151-9063-17c9e8974daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:12:48.665974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38151f9-a5c4-4151-9063-17c9e8974daa admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:12:48.666639 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38151f9-a5c4-4151-9063-17c9e8974daa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:12:48.668226 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38151f9-a5c4-4151-9063-17c9e8974daa admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:12:48.681052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 950/1879] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:12:48.685535 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d6af3-4929-43f1-9088-91e4da8889a0 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:12:48.696064 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d6af3-4929-43f1-9088-91e4da8889a0 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:12:48.700841 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4c083-a010-417a-9686-9b98333d6581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:12:48.701555 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4c083-a010-417a-9686-9b98333d6581 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:12:48.701836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4c083-a010-417a-9686-9b98333d6581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:12:48.702108 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4c083-a010-417a-9686-9b98333d6581 admin admin] PATH_INFO: `/v3/projects/31067798d7384f4489c25dd5660b823f/users/1baa5e519aac48a49dac5a452ae07694/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:12:48.716976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f4c083-a010-417a-9686-9b98333d6581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31067798d7384f4489c25dd5660b823f, user_id=1baa5e519aac48a49dac5a452ae07694, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:12:48.719625 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f4c083-a010-417a-9686-9b98333d6581 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:12:48.735830 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 930/1880] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:12:48 2022] PUT /identity/v3/projects/31067798d7384f4489c25dd5660b823f/users/1baa5e519aac48a49dac5a452ae07694/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:12:48.746383 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299a2c9-ae35-4bbd-ab87-d52cc6f40784 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:12:48.746659 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299a2c9-ae35-4bbd-ab87-d52cc6f40784 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:12:48.747206 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299a2c9-ae35-4bbd-ab87-d52cc6f40784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:12:48.779017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-b299a2c9-ae35-4bbd-ab87-d52cc6f40784 None None] MFA Rules not processed for user `1baa5e519aac48a49dac5a452ae07694`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:12:48.780290 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b299a2c9-ae35-4bbd-ab87-d52cc6f40784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:12:48.843649 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 951/1881] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:12:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:12:48.855913 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4c083-a010-417a-9686-9b98333d6581 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:12:48.857608 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-51f4c083-a010-417a-9686-9b98333d6581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:12:48.916403 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4c083-a010-417a-9686-9b98333d6581 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:12:48.920220 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250222c-7eef-4e5c-bb75-1cb2e6de9767 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dcfe5244bd04c00b6dde1d6f795d4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:12:48.920862 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250222c-7eef-4e5c-bb75-1cb2e6de9767 service glance] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:12:48.921145 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250222c-7eef-4e5c-bb75-1cb2e6de9767 service glance] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:12:48.921422 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250222c-7eef-4e5c-bb75-1cb2e6de9767 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:12:48.931723 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5250222c-7eef-4e5c-bb75-1cb2e6de9767 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:12:48.933239 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5250222c-7eef-4e5c-bb75-1cb2e6de9767 service glance] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:12:48.942093 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 931/1882] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:12:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:12:58.840736 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:12:58.938001 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:13:42.075458 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299a2c9-ae35-4bbd-ab87-d52cc6f40784 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:13:42.076921 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b299a2c9-ae35-4bbd-ab87-d52cc6f40784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:13:42.167003 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299a2c9-ae35-4bbd-ab87-d52cc6f40784 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:13:42.171940 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d77d4-bf14-4f74-aad1-db40e74491e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfbaaf32475a4c82a34948602d8f4e35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:13:42.172535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d77d4-bf14-4f74-aad1-db40e74491e7 service placement] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:13:42.172837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d77d4-bf14-4f74-aad1-db40e74491e7 service placement] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:13:42.173104 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d77d4-bf14-4f74-aad1-db40e74491e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:13:42.174789 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-0c8d77d4-bf14-4f74-aad1-db40e74491e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:13:42.185775 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d77d4-bf14-4f74-aad1-db40e74491e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:13:42.187331 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d77d4-bf14-4f74-aad1-db40e74491e7 service placement] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:13:42.196911 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 952/1883] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:13:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:13:52.192807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=41 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:14:01.604960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25dbe1a-0244-4b50-ab25-16fb4bfc0676 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:14:01.605511 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25dbe1a-0244-4b50-ab25-16fb4bfc0676 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:14:01.605511 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25dbe1a-0244-4b50-ab25-16fb4bfc0676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:14:01.634740 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c25dbe1a-0244-4b50-ab25-16fb4bfc0676 None None] MFA Rules not processed for user `7a59f547a20e48ee88e3e620ba1e87f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:14:01.635913 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c25dbe1a-0244-4b50-ab25-16fb4bfc0676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:14:01.696503 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 932/1884] 10.222.0.20 () {58 vars in 967 bytes} [Thu Oct 20 06:14:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:14:01.751460 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d77d4-bf14-4f74-aad1-db40e74491e7 service placement] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:14:01.752889 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-0c8d77d4-bf14-4f74-aad1-db40e74491e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:14:01.820436 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d77d4-bf14-4f74-aad1-db40e74491e7 service placement] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:14:01.825439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be59d06-66ac-46e1-bedf-75235c0b6cc6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeeab506ab5c4d6386fc899086f0abf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:14:01.826040 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be59d06-66ac-46e1-bedf-75235c0b6cc6 service swift] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:14:01.826316 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be59d06-66ac-46e1-bedf-75235c0b6cc6 service swift] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:14:01.826585 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be59d06-66ac-46e1-bedf-75235c0b6cc6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:14:01.834782 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be59d06-66ac-46e1-bedf-75235c0b6cc6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:14:01.836386 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be59d06-66ac-46e1-bedf-75235c0b6cc6 service swift] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:14:01.846997 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 953/1885] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Oct 20 06:14:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 06:14:03.275916 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25dbe1a-0244-4b50-ab25-16fb4bfc0676 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:14:03.277582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c25dbe1a-0244-4b50-ab25-16fb4bfc0676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:14:03.319569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b584d6-1a48-48c7-9fe3-6a4d09d35901 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:14:03.320085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b584d6-1a48-48c7-9fe3-6a4d09d35901 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:14:03.320365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b584d6-1a48-48c7-9fe3-6a4d09d35901 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:14:03.322638 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 954/1886] 10.222.0.20 () {54 vars in 912 bytes} [Thu Oct 20 06:14:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 06:14:03.327685 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a680d3a-8ee9-4c35-995e-25487dc33d0c None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:14:03.328315 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a680d3a-8ee9-4c35-995e-25487dc33d0c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:14:03.328656 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a680d3a-8ee9-4c35-995e-25487dc33d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:14:03.369791 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-9a680d3a-8ee9-4c35-995e-25487dc33d0c None None] MFA Rules not processed for user `cfbaaf32475a4c82a34948602d8f4e35`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:14:03.371126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9a680d3a-8ee9-4c35-995e-25487dc33d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:14:03.380648 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 955/1887] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Oct 20 06:14:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:14:03.391929 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25dbe1a-0244-4b50-ab25-16fb4bfc0676 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:14:03.398704 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907f4ee-1c7e-457b-b34f-2d1dc61c507f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:14:03.399328 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907f4ee-1c7e-457b-b34f-2d1dc61c507f service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:14:03.399934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907f4ee-1c7e-457b-b34f-2d1dc61c507f service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:14:03.400212 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907f4ee-1c7e-457b-b34f-2d1dc61c507f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:14:03.401990 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d907f4ee-1c7e-457b-b34f-2d1dc61c507f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:14:03.408676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a680d3a-8ee9-4c35-995e-25487dc33d0c None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:14:03.410033 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9a680d3a-8ee9-4c35-995e-25487dc33d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:14:03.415875 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d907f4ee-1c7e-457b-b34f-2d1dc61c507f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:14:03.417526 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d907f4ee-1c7e-457b-b34f-2d1dc61c507f service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:14:03.424560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a680d3a-8ee9-4c35-995e-25487dc33d0c None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:14:03.429597 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 933/1888] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:14:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:14:03.430355 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9205870f-b3e7-4695-80b3-b6e37a27d436 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfbaaf32475a4c82a34948602d8f4e35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:14:03.430961 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205870f-b3e7-4695-80b3-b6e37a27d436 service placement] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:14:03.431234 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205870f-b3e7-4695-80b3-b6e37a27d436 service placement] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:14:03.431501 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205870f-b3e7-4695-80b3-b6e37a27d436 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:14:03.440276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9205870f-b3e7-4695-80b3-b6e37a27d436 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:14:03.441882 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9205870f-b3e7-4695-80b3-b6e37a27d436 service placement] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:14:03.456965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 956/1889] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:14:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:14:05.587476 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b9fca-6592-4de0-a07b-a3548fa768e2 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:14:05.587984 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b9fca-6592-4de0-a07b-a3548fa768e2 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:14:05.588112 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b9fca-6592-4de0-a07b-a3548fa768e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:14:05.609476 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-425b9fca-6592-4de0-a07b-a3548fa768e2 None None] MFA Rules not processed for user `db34ffc8f90247daac1548bbb421b015`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:14:05.610684 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-425b9fca-6592-4de0-a07b-a3548fa768e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:14:05.659755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:14:05.668306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 934/1890] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:14:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:14:05.974712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9205870f-b3e7-4695-80b3-b6e37a27d436 service placement] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:14:05.976191 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9205870f-b3e7-4695-80b3-b6e37a27d436 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:14:06.055682 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9205870f-b3e7-4695-80b3-b6e37a27d436 service placement] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:14:06.059336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1732e9f-1659-4614-9c83-b6adca08c4b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:14:06.059940 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1732e9f-1659-4614-9c83-b6adca08c4b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:14:06.060218 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1732e9f-1659-4614-9c83-b6adca08c4b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:14:06.060490 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1732e9f-1659-4614-9c83-b6adca08c4b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:14:06.062603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f1732e9f-1659-4614-9c83-b6adca08c4b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:14:06.072786 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1732e9f-1659-4614-9c83-b6adca08c4b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:14:06.074229 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1732e9f-1659-4614-9c83-b6adca08c4b7 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:14:06.082803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 957/1891] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:14:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:14:06.410465 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b9fca-6592-4de0-a07b-a3548fa768e2 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:14:06.422988 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b9fca-6592-4de0-a07b-a3548fa768e2 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:14:06.428372 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ccf1a-8645-4b5a-87cd-ed9976d73efe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfbaaf32475a4c82a34948602d8f4e35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:14:06.429040 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ccf1a-8645-4b5a-87cd-ed9976d73efe service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:14:06.429316 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ccf1a-8645-4b5a-87cd-ed9976d73efe service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:14:06.429583 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ccf1a-8645-4b5a-87cd-ed9976d73efe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:14:06.431284 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-cf7ccf1a-8645-4b5a-87cd-ed9976d73efe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:14:06.444068 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ccf1a-8645-4b5a-87cd-ed9976d73efe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:14:06.445678 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ccf1a-8645-4b5a-87cd-ed9976d73efe service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:14:06.455616 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 935/1892] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:14:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:14:13.462620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1732e9f-1659-4614-9c83-b6adca08c4b7 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:14:13.478872 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1732e9f-1659-4614-9c83-b6adca08c4b7 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:14:13.486383 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f4c89-69e2-47e7-8bbe-e8cdee4736b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:14:13.487008 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f4c89-69e2-47e7-8bbe-e8cdee4736b7 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:14:13.487287 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f4c89-69e2-47e7-8bbe-e8cdee4736b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:14:13.487566 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f4c89-69e2-47e7-8bbe-e8cdee4736b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:14:13.489936 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6f9f4c89-69e2-47e7-8bbe-e8cdee4736b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:14:13.504397 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f4c89-69e2-47e7-8bbe-e8cdee4736b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:14:13.506232 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f4c89-69e2-47e7-8bbe-e8cdee4736b7 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:14:13.517334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 958/1893] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:14:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:14:16.451410 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=6 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:14:23.513278 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=88 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:14:33.027274 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ccf1a-8645-4b5a-87cd-ed9976d73efe service placement] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:14:33.029261 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-cf7ccf1a-8645-4b5a-87cd-ed9976d73efe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:14:33.044401 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ccf1a-8645-4b5a-87cd-ed9976d73efe service placement] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:14:33.049768 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc927db-94bf-40c4-add7-294d51441364 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:14:33.050353 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc927db-94bf-40c4-add7-294d51441364 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:14:33.050627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc927db-94bf-40c4-add7-294d51441364 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:14:33.050895 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc927db-94bf-40c4-add7-294d51441364 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:14:33.052526 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7bc927db-94bf-40c4-add7-294d51441364 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:14:33.131757 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc927db-94bf-40c4-add7-294d51441364 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:14:33.133360 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc927db-94bf-40c4-add7-294d51441364 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:14:33.152747 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 936/1894] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:14:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:14:40.035155 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f4c89-69e2-47e7-8bbe-e8cdee4736b7 service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:14:40.036532 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6f9f4c89-69e2-47e7-8bbe-e8cdee4736b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:14:40.114387 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f4c89-69e2-47e7-8bbe-e8cdee4736b7 service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:14:40.118546 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469dd532-73ca-46c1-99e9-58b3ca704229 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:14:40.119145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469dd532-73ca-46c1-99e9-58b3ca704229 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:14:40.119425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469dd532-73ca-46c1-99e9-58b3ca704229 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:14:40.119696 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469dd532-73ca-46c1-99e9-58b3ca704229 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:14:40.121430 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-469dd532-73ca-46c1-99e9-58b3ca704229 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:14:40.200992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469dd532-73ca-46c1-99e9-58b3ca704229 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:14:40.202799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469dd532-73ca-46c1-99e9-58b3ca704229 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:14:40.220979 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 959/1895] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:14:40 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:14:43.149337 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=41 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:14:46.653625 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc927db-94bf-40c4-add7-294d51441364 service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:14:46.665324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc927db-94bf-40c4-add7-294d51441364 service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:14:46.670709 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91f12f-fe86-4c05-b114-0d3069e4b944 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:14:46.671301 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91f12f-fe86-4c05-b114-0d3069e4b944 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:14:46.671576 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91f12f-fe86-4c05-b114-0d3069e4b944 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:14:46.671845 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91f12f-fe86-4c05-b114-0d3069e4b944 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:14:46.673537 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0e91f12f-fe86-4c05-b114-0d3069e4b944 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:14:46.751020 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91f12f-fe86-4c05-b114-0d3069e4b944 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:14:46.752641 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91f12f-fe86-4c05-b114-0d3069e4b944 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:14:46.771097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 937/1896] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:14:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:14:50.218314 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=70 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:14:52.947566 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469dd532-73ca-46c1-99e9-58b3ca704229 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:14:52.960250 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469dd532-73ca-46c1-99e9-58b3ca704229 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:14:52.966451 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fe690-0ca3-4600-961d-b083dc4eb2d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:14:52.967084 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fe690-0ca3-4600-961d-b083dc4eb2d3 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:14:52.967338 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fe690-0ca3-4600-961d-b083dc4eb2d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:14:52.967607 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fe690-0ca3-4600-961d-b083dc4eb2d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:14:52.969446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-228fe690-0ca3-4600-961d-b083dc4eb2d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:14:53.007231 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228fe690-0ca3-4600-961d-b083dc4eb2d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:14:53.008885 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228fe690-0ca3-4600-961d-b083dc4eb2d3 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:14:53.029926 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 960/1897] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:14:52 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:14:56.778385 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=41 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:15:03.027305 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=19 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:15:37.047011 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91f12f-fe86-4c05-b114-0d3069e4b944 service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:15:37.057521 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91f12f-fe86-4c05-b114-0d3069e4b944 service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:15:37.061214 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203edecd-b9b5-44ab-9912-99e98e55e2ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:15:37.061839 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203edecd-b9b5-44ab-9912-99e98e55e2ed service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:15:37.062115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203edecd-b9b5-44ab-9912-99e98e55e2ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:15:37.062386 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203edecd-b9b5-44ab-9912-99e98e55e2ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:15:37.064134 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-203edecd-b9b5-44ab-9912-99e98e55e2ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:15:37.148964 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203edecd-b9b5-44ab-9912-99e98e55e2ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:15:37.150424 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203edecd-b9b5-44ab-9912-99e98e55e2ed service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:15:37.160329 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 938/1898] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:15:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:15:42.122391 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fe690-0ca3-4600-961d-b083dc4eb2d3 service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:15:42.132996 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fe690-0ca3-4600-961d-b083dc4eb2d3 service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:15:42.137227 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbda6c3-a061-45cf-bb86-7dd9c4ea2da6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:15:42.137826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbda6c3-a061-45cf-bb86-7dd9c4ea2da6 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:15:42.138256 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbda6c3-a061-45cf-bb86-7dd9c4ea2da6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:15:42.138677 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbda6c3-a061-45cf-bb86-7dd9c4ea2da6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:15:42.141094 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6fbda6c3-a061-45cf-bb86-7dd9c4ea2da6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:15:42.154507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbda6c3-a061-45cf-bb86-7dd9c4ea2da6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:15:42.156106 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbda6c3-a061-45cf-bb86-7dd9c4ea2da6 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:15:42.165506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 961/1899] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:15:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:15:47.156068 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=37 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:15:52.162297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=6 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:16:20.939141 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203edecd-b9b5-44ab-9912-99e98e55e2ed service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:16:20.940766 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-203edecd-b9b5-44ab-9912-99e98e55e2ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:16:20.951997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203edecd-b9b5-44ab-9912-99e98e55e2ed service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:16:20.956515 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92112a32-cb58-47a8-91ea-216071fe4e71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:16:20.957264 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92112a32-cb58-47a8-91ea-216071fe4e71 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:16:20.957573 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92112a32-cb58-47a8-91ea-216071fe4e71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:16:20.957853 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92112a32-cb58-47a8-91ea-216071fe4e71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:16:20.959524 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-92112a32-cb58-47a8-91ea-216071fe4e71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:16:21.021560 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92112a32-cb58-47a8-91ea-216071fe4e71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:16:21.022995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92112a32-cb58-47a8-91ea-216071fe4e71 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:16:21.034338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 939/1900] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:16:20 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:16:31.029272 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=28 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:17:49.244247 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbda6c3-a061-45cf-bb86-7dd9c4ea2da6 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:17:49.253890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbda6c3-a061-45cf-bb86-7dd9c4ea2da6 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:17:49.257563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891e02a-e791-4661-a73f-ef20b683a82e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:17:49.258163 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891e02a-e791-4661-a73f-ef20b683a82e service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:17:49.258442 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891e02a-e791-4661-a73f-ef20b683a82e service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:17:49.258716 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891e02a-e791-4661-a73f-ef20b683a82e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:17:49.267633 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891e02a-e791-4661-a73f-ef20b683a82e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:17:49.269105 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891e02a-e791-4661-a73f-ef20b683a82e service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:17:49.278507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 962/1901] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:17:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:17:59.274538 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=6 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:18:42.461275 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92112a32-cb58-47a8-91ea-216071fe4e71 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:18:42.467252 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-92112a32-cb58-47a8-91ea-216071fe4e71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:18:42.573138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891e02a-e791-4661-a73f-ef20b683a82e service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:18:42.573716 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92112a32-cb58-47a8-91ea-216071fe4e71 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:18:42.574922 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9891e02a-e791-4661-a73f-ef20b683a82e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:18:42.578114 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9329bf67-7015-4cc5-9f39-35caa9a2149a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:18:42.578763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329bf67-7015-4cc5-9f39-35caa9a2149a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:42.579090 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329bf67-7015-4cc5-9f39-35caa9a2149a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:42.579375 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329bf67-7015-4cc5-9f39-35caa9a2149a admin admin] PATH_INFO: `/v3/users/ec11700e7fb44eb7ac9475d748c551f0` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:42.596357 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9329bf67-7015-4cc5-9f39-35caa9a2149a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec11700e7fb44eb7ac9475d748c551f0)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:18:42.598425 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9329bf67-7015-4cc5-9f39-35caa9a2149a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:18:42.603396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891e02a-e791-4661-a73f-ef20b683a82e service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:18:42.607434 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d8c0b-3f58-4ba1-9698-25f4e8c7e65a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:18:42.609606 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d8c0b-3f58-4ba1-9698-25f4e8c7e65a service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:42.610129 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d8c0b-3f58-4ba1-9698-25f4e8c7e65a service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:42.610623 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d8c0b-3f58-4ba1-9698-25f4e8c7e65a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:42.612605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-870d8c0b-3f58-4ba1-9698-25f4e8c7e65a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:18:42.634430 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d8c0b-3f58-4ba1-9698-25f4e8c7e65a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:18:42.637216 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d8c0b-3f58-4ba1-9698-25f4e8c7e65a service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:18:42.651483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 963/1902] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:18:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:18:42.663210 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9329bf67-7015-4cc5-9f39-35caa9a2149a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec11700e7fb44eb7ac9475d748c551f0'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:18:42.664091 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9329bf67-7015-4cc5-9f39-35caa9a2149a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec11700e7fb44eb7ac9475d748c551f0'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:18:42.676544 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9329bf67-7015-4cc5-9f39-35caa9a2149a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec11700e7fb44eb7ac9475d748c551f0'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:18:42.693985 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9329bf67-7015-4cc5-9f39-35caa9a2149a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec11700e7fb44eb7ac9475d748c551f0'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:18:42.701558 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9329bf67-7015-4cc5-9f39-35caa9a2149a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec11700e7fb44eb7ac9475d748c551f0'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:18:42.707931 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 940/1903] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:18:42 2022] DELETE /identity/v3/users/ec11700e7fb44eb7ac9475d748c551f0 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:18:42.713422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace3127-e83b-4615-a45d-71489e92aa78 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:42.713776 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace3127-e83b-4615-a45d-71489e92aa78 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:42.714154 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace3127-e83b-4615-a45d-71489e92aa78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:42.734108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-2ace3127-e83b-4615-a45d-71489e92aa78 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:18:42.736067 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2ace3127-e83b-4615-a45d-71489e92aa78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:18:42.804971 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 964/1904] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:18:42.814644 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9329bf67-7015-4cc5-9f39-35caa9a2149a admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:18:42.816043 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-9329bf67-7015-4cc5-9f39-35caa9a2149a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:18:42.873922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9329bf67-7015-4cc5-9f39-35caa9a2149a admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:18:42.877571 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c601b5-1308-42f4-b536-f2863d3a687b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:18:42.878175 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c601b5-1308-42f4-b536-f2863d3a687b service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:42.878449 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c601b5-1308-42f4-b536-f2863d3a687b service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:42.878716 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c601b5-1308-42f4-b536-f2863d3a687b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:42.888003 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c601b5-1308-42f4-b536-f2863d3a687b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:18:42.889454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c601b5-1308-42f4-b536-f2863d3a687b service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:18:42.899335 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 941/1905] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:18:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:18:43.186194 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace3127-e83b-4615-a45d-71489e92aa78 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:18:43.187626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2ace3127-e83b-4615-a45d-71489e92aa78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:18:43.200898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace3127-e83b-4615-a45d-71489e92aa78 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:18:43.206282 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f921e86-7236-42d0-a111-c52e77189f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:18:43.206282 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f921e86-7236-42d0-a111-c52e77189f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:43.206528 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f921e86-7236-42d0-a111-c52e77189f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:43.206733 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f921e86-7236-42d0-a111-c52e77189f0f admin admin] PATH_INFO: `/v3/projects/58a000d5925c4ab9a456d2c557c99523` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:43.218272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f921e86-7236-42d0-a111-c52e77189f0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=58a000d5925c4ab9a456d2c557c99523)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:18:43.220185 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f921e86-7236-42d0-a111-c52e77189f0f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:18:43.269519 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-7f921e86-7236-42d0-a111-c52e77189f0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58a000d5925c4ab9a456d2c557c99523', 'request_id': 'req-7f921e86-7236-42d0-a111-c52e77189f0f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:18:43.275576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-7f921e86-7236-42d0-a111-c52e77189f0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58a000d5925c4ab9a456d2c557c99523', 'request_id': 'req-7f921e86-7236-42d0-a111-c52e77189f0f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:18:43.282381 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-7f921e86-7236-42d0-a111-c52e77189f0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58a000d5925c4ab9a456d2c557c99523', 'request_id': 'req-7f921e86-7236-42d0-a111-c52e77189f0f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:18:43.296958 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-7f921e86-7236-42d0-a111-c52e77189f0f admin admin] The token cache is being invalidate because project 58a000d5925c4ab9a456d2c557c99523 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:18:43.297293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-7f921e86-7236-42d0-a111-c52e77189f0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:18:43.297917 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-7f921e86-7236-42d0-a111-c52e77189f0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:18:43.299676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 965/1906] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:18:43 2022] DELETE /identity/v3/projects/58a000d5925c4ab9a456d2c557c99523 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:18:43.306493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c6b8e-6dff-404d-b3a5-5272ba538e9f None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:43.306755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c6b8e-6dff-404d-b3a5-5272ba538e9f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:43.307027 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c6b8e-6dff-404d-b3a5-5272ba538e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:43.328114 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-223c6b8e-6dff-404d-b3a5-5272ba538e9f None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:18:43.329330 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-223c6b8e-6dff-404d-b3a5-5272ba538e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:18:43.334855 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:18:43.390668 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 942/1907] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:18:43.400937 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88854c17-e647-43db-88df-2785cd90fbb7 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:43.401223 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88854c17-e647-43db-88df-2785cd90fbb7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:43.401500 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88854c17-e647-43db-88df-2785cd90fbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:43.423044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-88854c17-e647-43db-88df-2785cd90fbb7 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:18:43.424309 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-88854c17-e647-43db-88df-2785cd90fbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:18:43.432366 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 966/1908] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:18:43.436777 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c6b8e-6dff-404d-b3a5-5272ba538e9f None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:18:43.448205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c6b8e-6dff-404d-b3a5-5272ba538e9f None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:18:43.452934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa877593-976e-403c-8653-e1eb4a321b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:18:43.461146 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa877593-976e-403c-8653-e1eb4a321b80 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:43.461618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa877593-976e-403c-8653-e1eb4a321b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:43.462176 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa877593-976e-403c-8653-e1eb4a321b80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:43.462689 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa877593-976e-403c-8653-e1eb4a321b80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:18:43.463061 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa877593-976e-403c-8653-e1eb4a321b80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:18:43.464807 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa877593-976e-403c-8653-e1eb4a321b80 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:18:43.478123 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 943/1909] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:18:43.482747 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88854c17-e647-43db-88df-2785cd90fbb7 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:18:43.493375 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88854c17-e647-43db-88df-2785cd90fbb7 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:18:43.498285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfede649-1255-410a-b435-d07930c394ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:18:43.498979 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfede649-1255-410a-b435-d07930c394ac admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:43.499195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfede649-1255-410a-b435-d07930c394ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:43.499432 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfede649-1255-410a-b435-d07930c394ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:43.500081 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfede649-1255-410a-b435-d07930c394ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:18:43.502161 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfede649-1255-410a-b435-d07930c394ac admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:18:43.545015 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 967/1910] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:18:43 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:18:43.549163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa877593-976e-403c-8653-e1eb4a321b80 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:18:43.559458 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa877593-976e-403c-8653-e1eb4a321b80 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:18:43.625156 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1461f-807c-462c-8d45-a0dba3c16b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:18:43.625775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1461f-807c-462c-8d45-a0dba3c16b4a admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:43.626051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1461f-807c-462c-8d45-a0dba3c16b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:43.626326 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1461f-807c-462c-8d45-a0dba3c16b4a admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:43.627119 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1461f-807c-462c-8d45-a0dba3c16b4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:18:43.629516 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1461f-807c-462c-8d45-a0dba3c16b4a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:18:43.654397 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 944/1911] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:18:43 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:18:43.668843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfede649-1255-410a-b435-d07930c394ac admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:18:43.679963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfede649-1255-410a-b435-d07930c394ac admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:18:43.684725 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80965fee-95e8-4d47-940b-135e269ab6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:18:43.685323 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80965fee-95e8-4d47-940b-135e269ab6e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:43.685597 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80965fee-95e8-4d47-940b-135e269ab6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:43.685862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80965fee-95e8-4d47-940b-135e269ab6e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:43.686494 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80965fee-95e8-4d47-940b-135e269ab6e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:18:43.688226 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80965fee-95e8-4d47-940b-135e269ab6e5 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:18:43.700736 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 968/1912] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:18:43.703551 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1461f-807c-462c-8d45-a0dba3c16b4a admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:18:43.714550 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1461f-807c-462c-8d45-a0dba3c16b4a admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:18:43.719264 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796faf2e-0dfd-4d6c-9b4f-c2e43d05c4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:18:43.719962 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796faf2e-0dfd-4d6c-9b4f-c2e43d05c4e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:43.720235 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796faf2e-0dfd-4d6c-9b4f-c2e43d05c4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:43.720499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796faf2e-0dfd-4d6c-9b4f-c2e43d05c4e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:43.721281 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796faf2e-0dfd-4d6c-9b4f-c2e43d05c4e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:18:43.722789 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796faf2e-0dfd-4d6c-9b4f-c2e43d05c4e3 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:18:43.740040 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 945/1913] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:18:43.741492 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80965fee-95e8-4d47-940b-135e269ab6e5 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:18:43.752752 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80965fee-95e8-4d47-940b-135e269ab6e5 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:18:43.758475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15641b50-4fc0-4c1c-9820-bccdbb011d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:18:43.759199 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15641b50-4fc0-4c1c-9820-bccdbb011d53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:43.759569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15641b50-4fc0-4c1c-9820-bccdbb011d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:43.759900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15641b50-4fc0-4c1c-9820-bccdbb011d53 admin admin] PATH_INFO: `/v3/projects/0cfa9834ec244713816c8775c3ba7f18/users/b2a67495335541dcb1cf1405ca9e3245/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:43.772000 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:18:43.776983 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15641b50-4fc0-4c1c-9820-bccdbb011d53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cfa9834ec244713816c8775c3ba7f18, user_id=b2a67495335541dcb1cf1405ca9e3245, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:18:43.779840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15641b50-4fc0-4c1c-9820-bccdbb011d53 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:18:43.795240 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 969/1914] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:18:43 2022] PUT /identity/v3/projects/0cfa9834ec244713816c8775c3ba7f18/users/b2a67495335541dcb1cf1405ca9e3245/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:18:43.800654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3f251-a637-4af8-941f-5a68adb65390 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:43.801048 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3f251-a637-4af8-941f-5a68adb65390 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:43.801554 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3f251-a637-4af8-941f-5a68adb65390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:43.822897 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-ebb3f251-a637-4af8-941f-5a68adb65390 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:18:43.824146 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ebb3f251-a637-4af8-941f-5a68adb65390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:18:43.890113 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 946/1915] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:18:43.899082 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15641b50-4fc0-4c1c-9820-bccdbb011d53 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:18:43.900868 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-15641b50-4fc0-4c1c-9820-bccdbb011d53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:18:43.973148 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15641b50-4fc0-4c1c-9820-bccdbb011d53 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:18:43.977860 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b98fc9-5fad-4358-8654-51cb7fcc05ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:18:43.978579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b98fc9-5fad-4358-8654-51cb7fcc05ea service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:43.978917 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b98fc9-5fad-4358-8654-51cb7fcc05ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:43.979253 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b98fc9-5fad-4358-8654-51cb7fcc05ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:43.997451 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b98fc9-5fad-4358-8654-51cb7fcc05ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:18:43.997451 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b98fc9-5fad-4358-8654-51cb7fcc05ea service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:18:44.003345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 970/1916] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:18:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:18:46.595681 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb3f251-a637-4af8-941f-5a68adb65390 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:18:46.597723 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ebb3f251-a637-4af8-941f-5a68adb65390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:18:46.622248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:18:46.723864 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb3f251-a637-4af8-941f-5a68adb65390 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:18:46.741604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a6a7b-8f9b-4e48-b83d-9b0bcf7b1571 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfbaaf32475a4c82a34948602d8f4e35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:18:46.742244 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a6a7b-8f9b-4e48-b83d-9b0bcf7b1571 service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:46.742526 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a6a7b-8f9b-4e48-b83d-9b0bcf7b1571 service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:46.742803 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a6a7b-8f9b-4e48-b83d-9b0bcf7b1571 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:46.747991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-150a6a7b-8f9b-4e48-b83d-9b0bcf7b1571 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:18:46.772282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a6a7b-8f9b-4e48-b83d-9b0bcf7b1571 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:18:46.778795 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a6a7b-8f9b-4e48-b83d-9b0bcf7b1571 service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:18:46.813539 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 947/1917] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:18:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:18:49.132879 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd023e-ed64-418e-9bb5-fe4f37a1de82 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:49.134322 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd023e-ed64-418e-9bb5-fe4f37a1de82 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:49.134869 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd023e-ed64-418e-9bb5-fe4f37a1de82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:49.228493 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d6bd023e-ed64-418e-9bb5-fe4f37a1de82 None None] MFA Rules not processed for user `b2a67495335541dcb1cf1405ca9e3245`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:18:49.246273 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d6bd023e-ed64-418e-9bb5-fe4f37a1de82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:18:49.343568 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 971/1918] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:18:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:18:49.348038 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a6a7b-8f9b-4e48-b83d-9b0bcf7b1571 service placement] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:18:49.361294 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a6a7b-8f9b-4e48-b83d-9b0bcf7b1571 service placement] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:18:49.366200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abeb70-a5b3-496d-8a5c-0a86d76a54cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:18:49.366812 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abeb70-a5b3-496d-8a5c-0a86d76a54cc admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:49.367090 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abeb70-a5b3-496d-8a5c-0a86d76a54cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:49.367359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abeb70-a5b3-496d-8a5c-0a86d76a54cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:49.368000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0abeb70-a5b3-496d-8a5c-0a86d76a54cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:18:49.370015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0abeb70-a5b3-496d-8a5c-0a86d76a54cc admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:18:49.438121 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 948/1919] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:18:49 2022] POST /identity/v3/projects => generated 373 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:18:49.439185 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd023e-ed64-418e-9bb5-fe4f37a1de82 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:18:49.452327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd023e-ed64-418e-9bb5-fe4f37a1de82 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:18:49.519492 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074bc2f-5a03-4b90-bffd-8ddf17b5e93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:18:49.520110 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074bc2f-5a03-4b90-bffd-8ddf17b5e93b admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:49.520390 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074bc2f-5a03-4b90-bffd-8ddf17b5e93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:49.521852 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074bc2f-5a03-4b90-bffd-8ddf17b5e93b admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:49.522541 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8074bc2f-5a03-4b90-bffd-8ddf17b5e93b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:18:49.524477 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8074bc2f-5a03-4b90-bffd-8ddf17b5e93b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:18:49.558085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 972/1920] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:18:49 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:18:49.558903 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abeb70-a5b3-496d-8a5c-0a86d76a54cc admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:18:49.569893 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abeb70-a5b3-496d-8a5c-0a86d76a54cc admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:18:49.574642 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e2cac-a982-44a0-a906-a26e7381bb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:18:49.575258 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e2cac-a982-44a0-a906-a26e7381bb90 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:49.575531 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e2cac-a982-44a0-a906-a26e7381bb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:49.575833 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e2cac-a982-44a0-a906-a26e7381bb90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:49.576475 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e2cac-a982-44a0-a906-a26e7381bb90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:18:49.578263 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e2cac-a982-44a0-a906-a26e7381bb90 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:18:49.592764 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 949/1921] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:18:49.596813 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074bc2f-5a03-4b90-bffd-8ddf17b5e93b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:18:49.605429 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074bc2f-5a03-4b90-bffd-8ddf17b5e93b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:18:49.610751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683572ef-725a-4897-9a06-c4c03c299b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:18:49.611539 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683572ef-725a-4897-9a06-c4c03c299b80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:49.611895 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683572ef-725a-4897-9a06-c4c03c299b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:49.612233 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683572ef-725a-4897-9a06-c4c03c299b80 admin admin] PATH_INFO: `/v3/projects/47e2a0ec4763475fa1a047220baea2c7/users/639e084a25f04167afff8739730e6d71/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:49.623100 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:18:49.637000 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683572ef-725a-4897-9a06-c4c03c299b80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47e2a0ec4763475fa1a047220baea2c7, user_id=639e084a25f04167afff8739730e6d71, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:18:49.641548 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683572ef-725a-4897-9a06-c4c03c299b80 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:18:49.657978 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 973/1922] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:18:49 2022] PUT /identity/v3/projects/47e2a0ec4763475fa1a047220baea2c7/users/639e084a25f04167afff8739730e6d71/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:18:49.662051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e2cac-a982-44a0-a906-a26e7381bb90 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:18:49.673090 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e2cac-a982-44a0-a906-a26e7381bb90 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:18:49.743941 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d513b4-cb64-4687-bfaf-8816fc483602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:18:49.744319 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d513b4-cb64-4687-bfaf-8816fc483602 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:49.744604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d513b4-cb64-4687-bfaf-8816fc483602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:49.744908 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d513b4-cb64-4687-bfaf-8816fc483602 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:49.745563 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d513b4-cb64-4687-bfaf-8816fc483602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:18:49.748963 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d513b4-cb64-4687-bfaf-8816fc483602 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:18:49.765473 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 950/1923] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:18:49.766804 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683572ef-725a-4897-9a06-c4c03c299b80 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:18:49.778456 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683572ef-725a-4897-9a06-c4c03c299b80 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:18:49.783438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271acb7-32d2-4000-bd79-f9e5aec86b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:18:49.784138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271acb7-32d2-4000-bd79-f9e5aec86b37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:49.784420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271acb7-32d2-4000-bd79-f9e5aec86b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:49.784738 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271acb7-32d2-4000-bd79-f9e5aec86b37 admin admin] PATH_INFO: `/v3/projects/47e2a0ec4763475fa1a047220baea2c7/users/639e084a25f04167afff8739730e6d71/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:49.786913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271acb7-32d2-4000-bd79-f9e5aec86b37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47e2a0ec4763475fa1a047220baea2c7, user_id=639e084a25f04167afff8739730e6d71, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:18:49.790082 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271acb7-32d2-4000-bd79-f9e5aec86b37 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:18:49.805819 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 974/1924] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:18:49 2022] PUT /identity/v3/projects/47e2a0ec4763475fa1a047220baea2c7/users/639e084a25f04167afff8739730e6d71/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:18:56.369533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2c491-99ad-4e69-a02c-c1a58d1bf213 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:56.370355 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2c491-99ad-4e69-a02c-c1a58d1bf213 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:56.370632 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2c491-99ad-4e69-a02c-c1a58d1bf213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:56.406588 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c0e2c491-99ad-4e69-a02c-c1a58d1bf213 None None] MFA Rules not processed for user `639e084a25f04167afff8739730e6d71`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:18:56.407940 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c0e2c491-99ad-4e69-a02c-c1a58d1bf213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:18:56.552486 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:18:56.565803 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 951/1925] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:18:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:18:56.584647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271acb7-32d2-4000-bd79-f9e5aec86b37 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:18:56.612011 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-0271acb7-32d2-4000-bd79-f9e5aec86b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:18:56.812440 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271acb7-32d2-4000-bd79-f9e5aec86b37 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:18:56.826563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc83c02-4300-4574-8ecb-81a2ee70eefc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:18:56.827203 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc83c02-4300-4574-8ecb-81a2ee70eefc service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:56.827582 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc83c02-4300-4574-8ecb-81a2ee70eefc service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:56.827783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc83c02-4300-4574-8ecb-81a2ee70eefc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:56.923351 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc83c02-4300-4574-8ecb-81a2ee70eefc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:18:56.925639 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc83c02-4300-4574-8ecb-81a2ee70eefc service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:18:56.936120 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 975/1926] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:18:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:18:59.232104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938a62f-89f6-4b30-be34-c1c1173916dc None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:18:59.232659 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938a62f-89f6-4b30-be34-c1c1173916dc None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:18:59.232947 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938a62f-89f6-4b30-be34-c1c1173916dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:18:59.254322 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-a938a62f-89f6-4b30-be34-c1c1173916dc None None] MFA Rules not processed for user `b2a67495335541dcb1cf1405ca9e3245`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:18:59.256079 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a938a62f-89f6-4b30-be34-c1c1173916dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:18:59.263446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 952/1927] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:18:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:19:06.931525 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=66,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:19:09.248446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=5 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:19:34.012397 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc83c02-4300-4574-8ecb-81a2ee70eefc service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:19:34.014017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5fc83c02-4300-4574-8ecb-81a2ee70eefc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:19:34.073148 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc83c02-4300-4574-8ecb-81a2ee70eefc service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:19:34.076734 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9f412-a004-4429-a7bc-bd88982b9d19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:19:34.077350 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9f412-a004-4429-a7bc-bd88982b9d19 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:19:34.077624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9f412-a004-4429-a7bc-bd88982b9d19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:19:34.077888 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9f412-a004-4429-a7bc-bd88982b9d19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:19:34.079590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-99c9f412-a004-4429-a7bc-bd88982b9d19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:19:34.138735 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9f412-a004-4429-a7bc-bd88982b9d19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:19:34.140173 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9f412-a004-4429-a7bc-bd88982b9d19 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:19:34.149550 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 976/1928] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:19:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:19:40.608909 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938a62f-89f6-4b30-be34-c1c1173916dc None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:19:40.614916 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938a62f-89f6-4b30-be34-c1c1173916dc None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:19:40.619382 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0001a04-0a4c-47f5-8517-fd89549b2097 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:19:40.620034 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0001a04-0a4c-47f5-8517-fd89549b2097 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:19:40.620312 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0001a04-0a4c-47f5-8517-fd89549b2097 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:19:40.620582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0001a04-0a4c-47f5-8517-fd89549b2097 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:19:40.622351 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b0001a04-0a4c-47f5-8517-fd89549b2097 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:19:40.685064 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0001a04-0a4c-47f5-8517-fd89549b2097 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:19:40.686592 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0001a04-0a4c-47f5-8517-fd89549b2097 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:19:40.696815 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 953/1929] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:19:40 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:19:44.144990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=48 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:19:46.653337 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9f412-a004-4429-a7bc-bd88982b9d19 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:19:46.666395 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9f412-a004-4429-a7bc-bd88982b9d19 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:19:46.671855 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7eeee-d3e3-4bfd-8d74-8532bc66ce06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:19:46.672472 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7eeee-d3e3-4bfd-8d74-8532bc66ce06 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:19:46.672776 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7eeee-d3e3-4bfd-8d74-8532bc66ce06 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:19:46.673051 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7eeee-d3e3-4bfd-8d74-8532bc66ce06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:19:46.674772 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-27f7eeee-d3e3-4bfd-8d74-8532bc66ce06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:19:46.740674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7eeee-d3e3-4bfd-8d74-8532bc66ce06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:19:46.742545 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7eeee-d3e3-4bfd-8d74-8532bc66ce06 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:19:46.752849 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 977/1930] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:19:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:19:50.691722 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=28 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:19:52.622697 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0001a04-0a4c-47f5-8517-fd89549b2097 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:19:52.633010 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0001a04-0a4c-47f5-8517-fd89549b2097 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:19:52.637448 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0c2fd-aecc-47c5-946a-073533748842 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:19:52.638067 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0c2fd-aecc-47c5-946a-073533748842 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:19:52.638345 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0c2fd-aecc-47c5-946a-073533748842 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:19:52.638618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0c2fd-aecc-47c5-946a-073533748842 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:19:52.640372 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-9fd0c2fd-aecc-47c5-946a-073533748842 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:19:52.705229 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0c2fd-aecc-47c5-946a-073533748842 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:19:52.706686 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0c2fd-aecc-47c5-946a-073533748842 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:19:52.716082 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 954/1931] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:19:52 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:19:56.748161 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=28 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:20:02.711630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=28 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:20:33.821912 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7eeee-d3e3-4bfd-8d74-8532bc66ce06 service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:20:33.884449 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7eeee-d3e3-4bfd-8d74-8532bc66ce06 service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:20:33.888061 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c360f-f2b5-4ee1-a412-59ca89089c55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:20:33.888663 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c360f-f2b5-4ee1-a412-59ca89089c55 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:20:33.888966 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c360f-f2b5-4ee1-a412-59ca89089c55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:20:33.889235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c360f-f2b5-4ee1-a412-59ca89089c55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:20:33.890971 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-175c360f-f2b5-4ee1-a412-59ca89089c55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:20:33.905567 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c360f-f2b5-4ee1-a412-59ca89089c55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:20:33.907238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c360f-f2b5-4ee1-a412-59ca89089c55 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:20:33.919173 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 978/1932] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:20:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:20:35.449344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0c2fd-aecc-47c5-946a-073533748842 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:20:35.463046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0c2fd-aecc-47c5-946a-073533748842 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:20:35.474729 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d36fad-c01a-41f6-a021-6a3166fd31ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:20:35.475425 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d36fad-c01a-41f6-a021-6a3166fd31ff service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:20:35.475917 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d36fad-c01a-41f6-a021-6a3166fd31ff service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:20:35.476351 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d36fad-c01a-41f6-a021-6a3166fd31ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:20:35.479169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c4d36fad-c01a-41f6-a021-6a3166fd31ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:20:35.497717 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d36fad-c01a-41f6-a021-6a3166fd31ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:20:35.499611 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d36fad-c01a-41f6-a021-6a3166fd31ff service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:20:35.524992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 955/1933] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:20:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:20:37.514885 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c360f-f2b5-4ee1-a412-59ca89089c55 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:20:37.516244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-175c360f-f2b5-4ee1-a412-59ca89089c55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:20:37.531898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c360f-f2b5-4ee1-a412-59ca89089c55 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:20:37.537567 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a797665-753a-47cf-8e8d-73b5e30822c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:20:37.538196 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a797665-753a-47cf-8e8d-73b5e30822c9 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:20:37.538473 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a797665-753a-47cf-8e8d-73b5e30822c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:20:37.538740 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a797665-753a-47cf-8e8d-73b5e30822c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:20:37.541265 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7a797665-753a-47cf-8e8d-73b5e30822c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:20:37.604833 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a797665-753a-47cf-8e8d-73b5e30822c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:20:37.606478 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a797665-753a-47cf-8e8d-73b5e30822c9 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:20:37.627360 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 979/1934] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:20:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:20:45.506941 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=6 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:20:46.117099 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d36fad-c01a-41f6-a021-6a3166fd31ff service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:20:46.127265 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d36fad-c01a-41f6-a021-6a3166fd31ff service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:20:46.130889 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab323a0e-6a8f-4e70-a9f0-f9af80188308 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:20:46.131494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab323a0e-6a8f-4e70-a9f0-f9af80188308 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:20:46.132027 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab323a0e-6a8f-4e70-a9f0-f9af80188308 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:20:46.132294 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab323a0e-6a8f-4e70-a9f0-f9af80188308 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:20:46.134035 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ab323a0e-6a8f-4e70-a9f0-f9af80188308 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:20:46.146585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab323a0e-6a8f-4e70-a9f0-f9af80188308 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:20:46.148053 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab323a0e-6a8f-4e70-a9f0-f9af80188308 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:20:46.156616 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 956/1935] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:20:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:20:47.623879 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=58 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:20:56.153020 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=6 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:23:42.780471 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a797665-753a-47cf-8e8d-73b5e30822c9 service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:23:42.781984 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7a797665-753a-47cf-8e8d-73b5e30822c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:23:42.823818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a797665-753a-47cf-8e8d-73b5e30822c9 service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:23:42.827449 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52cc4b1-b29d-45b6-850a-bf34fcddb421 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:23:42.828054 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cc4b1-b29d-45b6-850a-bf34fcddb421 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:23:42.828330 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cc4b1-b29d-45b6-850a-bf34fcddb421 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:23:42.828598 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cc4b1-b29d-45b6-850a-bf34fcddb421 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:23:42.830333 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f52cc4b1-b29d-45b6-850a-bf34fcddb421 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:23:42.865642 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52cc4b1-b29d-45b6-850a-bf34fcddb421 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:23:42.867084 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52cc4b1-b29d-45b6-850a-bf34fcddb421 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:23:42.878093 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 980/1936] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:23:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:23:47.410077 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab323a0e-6a8f-4e70-a9f0-f9af80188308 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:23:47.411587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ab323a0e-6a8f-4e70-a9f0-f9af80188308 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:23:47.532797 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab323a0e-6a8f-4e70-a9f0-f9af80188308 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:23:47.538451 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0025c3-2e85-44ab-960e-66193c2a8916 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfbaaf32475a4c82a34948602d8f4e35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:23:47.539062 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0025c3-2e85-44ab-960e-66193c2a8916 service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:23:47.539344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0025c3-2e85-44ab-960e-66193c2a8916 service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:23:47.539618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0025c3-2e85-44ab-960e-66193c2a8916 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:23:47.548263 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0025c3-2e85-44ab-960e-66193c2a8916 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:23:47.549867 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0025c3-2e85-44ab-960e-66193c2a8916 service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:23:47.560506 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 957/1937] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:23:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:23:52.873292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=28 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:23:57.561320 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=54 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:24:07.345360 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52cc4b1-b29d-45b6-850a-bf34fcddb421 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:24:07.354546 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f52cc4b1-b29d-45b6-850a-bf34fcddb421 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:24:07.365824 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52cc4b1-b29d-45b6-850a-bf34fcddb421 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:24:07.370050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e2420-59b8-4420-93eb-5f2ae5ad716e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:24:07.370659 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e2420-59b8-4420-93eb-5f2ae5ad716e service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:24:07.370936 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e2420-59b8-4420-93eb-5f2ae5ad716e service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:24:07.371204 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e2420-59b8-4420-93eb-5f2ae5ad716e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:24:07.372963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-826e2420-59b8-4420-93eb-5f2ae5ad716e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:24:07.455256 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e2420-59b8-4420-93eb-5f2ae5ad716e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:24:07.456688 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e2420-59b8-4420-93eb-5f2ae5ad716e service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:24:07.474555 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 981/1938] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:24:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:24:07.556286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0025c3-2e85-44ab-960e-66193c2a8916 service placement] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:24:07.565697 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0025c3-2e85-44ab-960e-66193c2a8916 service placement] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:24:07.569210 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02c088-04f3-48ae-9fd9-03caa89b7ca2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:24:07.569806 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02c088-04f3-48ae-9fd9-03caa89b7ca2 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:24:07.570678 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02c088-04f3-48ae-9fd9-03caa89b7ca2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:24:07.571031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02c088-04f3-48ae-9fd9-03caa89b7ca2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:24:07.579414 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af02c088-04f3-48ae-9fd9-03caa89b7ca2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:24:07.580850 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af02c088-04f3-48ae-9fd9-03caa89b7ca2 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:24:07.590239 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 958/1939] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:24:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:24:17.471977 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=41 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:24:17.586378 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=6 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:24:32.933805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e2420-59b8-4420-93eb-5f2ae5ad716e service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:24:32.964429 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e2420-59b8-4420-93eb-5f2ae5ad716e service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:24:32.970226 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edeaed4-ae6d-4b47-a0c2-d1c9400b06f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:24:32.970843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edeaed4-ae6d-4b47-a0c2-d1c9400b06f9 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:24:32.971124 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edeaed4-ae6d-4b47-a0c2-d1c9400b06f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:24:32.971397 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edeaed4-ae6d-4b47-a0c2-d1c9400b06f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:24:32.980073 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edeaed4-ae6d-4b47-a0c2-d1c9400b06f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:24:32.981716 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edeaed4-ae6d-4b47-a0c2-d1c9400b06f9 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:24:32.990889 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 982/1940] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:24:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:24:42.986631 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=13 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:25:35.074873 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02c088-04f3-48ae-9fd9-03caa89b7ca2 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:25:35.095636 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02c088-04f3-48ae-9fd9-03caa89b7ca2 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:25:35.099232 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad2725-adb7-4975-9e09-8db7627ef581 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:25:35.099824 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad2725-adb7-4975-9e09-8db7627ef581 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:25:35.100097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad2725-adb7-4975-9e09-8db7627ef581 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:25:35.100362 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad2725-adb7-4975-9e09-8db7627ef581 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:25:35.110548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad2725-adb7-4975-9e09-8db7627ef581 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:25:35.111960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad2725-adb7-4975-9e09-8db7627ef581 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:25:35.122627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 959/1941] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:25:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:25:45.117886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=10 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:25:52.112539 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edeaed4-ae6d-4b47-a0c2-d1c9400b06f9 service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:25:52.122529 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edeaed4-ae6d-4b47-a0c2-d1c9400b06f9 service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:25:52.126136 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0396f-dc92-4c0c-9f15-28bba7dbf8ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:25:52.126737 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0396f-dc92-4c0c-9f15-28bba7dbf8ba service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:25:52.127014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0396f-dc92-4c0c-9f15-28bba7dbf8ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:25:52.127287 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0396f-dc92-4c0c-9f15-28bba7dbf8ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:25:52.135766 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d0396f-dc92-4c0c-9f15-28bba7dbf8ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:25:52.137239 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d0396f-dc92-4c0c-9f15-28bba7dbf8ba service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:25:52.145346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 983/1942] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:25:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:26:02.141711 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=6 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:26:07.908540 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad2725-adb7-4975-9e09-8db7627ef581 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:07.928731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad2725-adb7-4975-9e09-8db7627ef581 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:07.932602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462f847-b9bd-4a1e-9ab5-d8da5cf9d9b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:07.933256 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462f847-b9bd-4a1e-9ab5-d8da5cf9d9b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:07.933536 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462f847-b9bd-4a1e-9ab5-d8da5cf9d9b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:07.933811 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462f847-b9bd-4a1e-9ab5-d8da5cf9d9b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:07.942762 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5462f847-b9bd-4a1e-9ab5-d8da5cf9d9b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:07.944896 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5462f847-b9bd-4a1e-9ab5-d8da5cf9d9b3 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:07.972642 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 960/1943] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:26:07 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:26:08.054339 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0396f-dc92-4c0c-9f15-28bba7dbf8ba service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:08.063812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0396f-dc92-4c0c-9f15-28bba7dbf8ba service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:08.069539 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe071b8-97a6-4afb-9a3d-4ceacbba5fbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:08.070149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe071b8-97a6-4afb-9a3d-4ceacbba5fbd service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:08.070427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe071b8-97a6-4afb-9a3d-4ceacbba5fbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:08.070720 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe071b8-97a6-4afb-9a3d-4ceacbba5fbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:08.079536 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe071b8-97a6-4afb-9a3d-4ceacbba5fbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:08.081639 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe071b8-97a6-4afb-9a3d-4ceacbba5fbd service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:08.109766 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 984/1944] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:26:08 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:26:08.189385 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462f847-b9bd-4a1e-9ab5-d8da5cf9d9b3 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:08.198105 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462f847-b9bd-4a1e-9ab5-d8da5cf9d9b3 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:08.201680 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc68d9-03e7-4fd0-88b7-b506b759e1ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:08.202272 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc68d9-03e7-4fd0-88b7-b506b759e1ac service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:08.202545 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc68d9-03e7-4fd0-88b7-b506b759e1ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:08.202812 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc68d9-03e7-4fd0-88b7-b506b759e1ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:08.211459 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc68d9-03e7-4fd0-88b7-b506b759e1ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:08.212899 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc68d9-03e7-4fd0-88b7-b506b759e1ac service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:08.239799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 961/1945] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:26:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:26:08.289331 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe071b8-97a6-4afb-9a3d-4ceacbba5fbd service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:08.304564 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe071b8-97a6-4afb-9a3d-4ceacbba5fbd service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:08.308251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62ed9a-1b07-4dbe-9258-c35d7e221b79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:08.308893 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62ed9a-1b07-4dbe-9258-c35d7e221b79 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:08.309171 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62ed9a-1b07-4dbe-9258-c35d7e221b79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:08.309465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62ed9a-1b07-4dbe-9258-c35d7e221b79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:08.318349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62ed9a-1b07-4dbe-9258-c35d7e221b79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:08.319889 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62ed9a-1b07-4dbe-9258-c35d7e221b79 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:08.329260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 985/1946] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:26:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:26:09.882949 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc68d9-03e7-4fd0-88b7-b506b759e1ac service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:09.899064 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc68d9-03e7-4fd0-88b7-b506b759e1ac service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:09.905363 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce30e3-a117-40b8-8117-e670f5f84767 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:09.906312 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce30e3-a117-40b8-8117-e670f5f84767 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:09.906786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce30e3-a117-40b8-8117-e670f5f84767 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:09.907225 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce30e3-a117-40b8-8117-e670f5f84767 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:09.920221 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce30e3-a117-40b8-8117-e670f5f84767 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:09.922423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce30e3-a117-40b8-8117-e670f5f84767 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:09.945466 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 962/1947] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:26:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:26:13.650992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62ed9a-1b07-4dbe-9258-c35d7e221b79 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:13.660358 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62ed9a-1b07-4dbe-9258-c35d7e221b79 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:13.663965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9157b9f-193d-46f9-b2e3-2fd25d6562b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:13.664561 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9157b9f-193d-46f9-b2e3-2fd25d6562b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:13.664931 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9157b9f-193d-46f9-b2e3-2fd25d6562b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:13.665208 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9157b9f-193d-46f9-b2e3-2fd25d6562b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:13.667026 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d9157b9f-193d-46f9-b2e3-2fd25d6562b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:13.755112 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9157b9f-193d-46f9-b2e3-2fd25d6562b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:13.757255 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9157b9f-193d-46f9-b2e3-2fd25d6562b0 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:13.779667 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 986/1948] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:26:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:26:17.312122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce30e3-a117-40b8-8117-e670f5f84767 service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:17.340066 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce30e3-a117-40b8-8117-e670f5f84767 service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:17.350440 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30810863-9524-4969-b0f6-535a9c8edfad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfbaaf32475a4c82a34948602d8f4e35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:17.351323 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30810863-9524-4969-b0f6-535a9c8edfad service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:17.352472 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30810863-9524-4969-b0f6-535a9c8edfad service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:17.353217 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30810863-9524-4969-b0f6-535a9c8edfad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:17.355486 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-30810863-9524-4969-b0f6-535a9c8edfad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:17.374522 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30810863-9524-4969-b0f6-535a9c8edfad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:17.378647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30810863-9524-4969-b0f6-535a9c8edfad service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:17.379489 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9157b9f-193d-46f9-b2e3-2fd25d6562b0 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:17.396000 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d9157b9f-193d-46f9-b2e3-2fd25d6562b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:17.408726 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 963/1949] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:26:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:26:17.409864 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9157b9f-193d-46f9-b2e3-2fd25d6562b0 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:17.415591 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66391b58-1cfd-4f8c-9e47-a1a0f89a8c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:17.416464 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66391b58-1cfd-4f8c-9e47-a1a0f89a8c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:17.416988 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66391b58-1cfd-4f8c-9e47-a1a0f89a8c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:17.417524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66391b58-1cfd-4f8c-9e47-a1a0f89a8c47 admin admin] PATH_INFO: `/v3/users/57d4cbd2a2154e5d925462df7c77815a` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:17.436142 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66391b58-1cfd-4f8c-9e47-a1a0f89a8c47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57d4cbd2a2154e5d925462df7c77815a)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:17.438052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66391b58-1cfd-4f8c-9e47-a1a0f89a8c47 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:17.490010 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-66391b58-1cfd-4f8c-9e47-a1a0f89a8c47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57d4cbd2a2154e5d925462df7c77815a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:17.491278 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-66391b58-1cfd-4f8c-9e47-a1a0f89a8c47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57d4cbd2a2154e5d925462df7c77815a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:17.509292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-66391b58-1cfd-4f8c-9e47-a1a0f89a8c47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57d4cbd2a2154e5d925462df7c77815a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:17.524926 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:26:17.527534 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-66391b58-1cfd-4f8c-9e47-a1a0f89a8c47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57d4cbd2a2154e5d925462df7c77815a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:17.528475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-66391b58-1cfd-4f8c-9e47-a1a0f89a8c47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57d4cbd2a2154e5d925462df7c77815a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:17.547362 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 987/1950] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:26:17 2022] DELETE /identity/v3/users/57d4cbd2a2154e5d925462df7c77815a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:26:17.760331 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30810863-9524-4969-b0f6-535a9c8edfad service placement] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:17.771830 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30810863-9524-4969-b0f6-535a9c8edfad service placement] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:17.845126 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62783841-e975-4f4b-b008-6fb69ebcef65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:17.845765 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62783841-e975-4f4b-b008-6fb69ebcef65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:17.846046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62783841-e975-4f4b-b008-6fb69ebcef65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:17.846319 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62783841-e975-4f4b-b008-6fb69ebcef65 admin admin] PATH_INFO: `/v3/projects/9f0d80988ca941acb8ad4318338dfbea` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:17.847462 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62783841-e975-4f4b-b008-6fb69ebcef65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f0d80988ca941acb8ad4318338dfbea)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:17.849685 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62783841-e975-4f4b-b008-6fb69ebcef65 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:17.898835 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-62783841-e975-4f4b-b008-6fb69ebcef65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f0d80988ca941acb8ad4318338dfbea', 'request_id': 'req-62783841-e975-4f4b-b008-6fb69ebcef65'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:17.905705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-62783841-e975-4f4b-b008-6fb69ebcef65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f0d80988ca941acb8ad4318338dfbea', 'request_id': 'req-62783841-e975-4f4b-b008-6fb69ebcef65'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:17.912938 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-62783841-e975-4f4b-b008-6fb69ebcef65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f0d80988ca941acb8ad4318338dfbea', 'request_id': 'req-62783841-e975-4f4b-b008-6fb69ebcef65'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:17.921436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:26:17.927200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-62783841-e975-4f4b-b008-6fb69ebcef65 admin admin] The token cache is being invalidate because project 9f0d80988ca941acb8ad4318338dfbea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:26:17.927743 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-62783841-e975-4f4b-b008-6fb69ebcef65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:17.928951 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-62783841-e975-4f4b-b008-6fb69ebcef65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:17.930965 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 964/1951] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:26:17 2022] DELETE /identity/v3/projects/9f0d80988ca941acb8ad4318338dfbea => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:26:17.936633 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbe8b2-6ec4-4a9b-a897-f59710d247e2 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:17.936941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbe8b2-6ec4-4a9b-a897-f59710d247e2 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:17.937241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbe8b2-6ec4-4a9b-a897-f59710d247e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:17.996466 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-e5fbe8b2-6ec4-4a9b-a897-f59710d247e2 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:26:17.997683 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e5fbe8b2-6ec4-4a9b-a897-f59710d247e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:18.057971 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 988/1952] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:26:18.065861 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e29017-8c7b-4327-89ec-e0e62ab9e12e None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:18.066245 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e29017-8c7b-4327-89ec-e0e62ab9e12e None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:18.066522 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e29017-8c7b-4327-89ec-e0e62ab9e12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:18.087982 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-75e29017-8c7b-4327-89ec-e0e62ab9e12e None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:26:18.089224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-75e29017-8c7b-4327-89ec-e0e62ab9e12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:18.097233 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 965/1953] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:26:18.101844 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbe8b2-6ec4-4a9b-a897-f59710d247e2 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:18.112143 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbe8b2-6ec4-4a9b-a897-f59710d247e2 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:18.117431 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad716b4-5362-48a4-87f0-d3af4e9dabaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:18.118043 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad716b4-5362-48a4-87f0-d3af4e9dabaf admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:18.118318 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad716b4-5362-48a4-87f0-d3af4e9dabaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:18.118586 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad716b4-5362-48a4-87f0-d3af4e9dabaf admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:18.119228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad716b4-5362-48a4-87f0-d3af4e9dabaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:26:18.119531 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad716b4-5362-48a4-87f0-d3af4e9dabaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:18.121159 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad716b4-5362-48a4-87f0-d3af4e9dabaf admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:18.135493 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 989/1954] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:26:18.139968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e29017-8c7b-4327-89ec-e0e62ab9e12e None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:18.149536 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e29017-8c7b-4327-89ec-e0e62ab9e12e None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:18.154222 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505df289-e6a0-48bd-bb82-339cb640ee37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:18.154825 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505df289-e6a0-48bd-bb82-339cb640ee37 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:18.155106 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505df289-e6a0-48bd-bb82-339cb640ee37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:18.155382 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505df289-e6a0-48bd-bb82-339cb640ee37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:18.156081 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505df289-e6a0-48bd-bb82-339cb640ee37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:18.158038 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505df289-e6a0-48bd-bb82-339cb640ee37 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:18.187436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 966/1955] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:26:18 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:26:18.191606 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad716b4-5362-48a4-87f0-d3af4e9dabaf admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:18.202322 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad716b4-5362-48a4-87f0-d3af4e9dabaf admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:18.259121 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89eb868-8e75-4832-b918-1cdaae8cc539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:18.259998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89eb868-8e75-4832-b918-1cdaae8cc539 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:18.260272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89eb868-8e75-4832-b918-1cdaae8cc539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:18.260538 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89eb868-8e75-4832-b918-1cdaae8cc539 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:18.261254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89eb868-8e75-4832-b918-1cdaae8cc539 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:18.263134 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89eb868-8e75-4832-b918-1cdaae8cc539 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:18.287858 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 990/1956] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:26:18 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:26:18.289467 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505df289-e6a0-48bd-bb82-339cb640ee37 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:18.306564 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505df289-e6a0-48bd-bb82-339cb640ee37 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:18.312059 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137ee4c-2196-4390-9dec-951c8ab48f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:18.312803 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137ee4c-2196-4390-9dec-951c8ab48f96 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:18.313337 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137ee4c-2196-4390-9dec-951c8ab48f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:18.313442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137ee4c-2196-4390-9dec-951c8ab48f96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:18.314099 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b137ee4c-2196-4390-9dec-951c8ab48f96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:18.315669 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b137ee4c-2196-4390-9dec-951c8ab48f96 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:18.330792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 967/1957] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:26:18.332046 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89eb868-8e75-4832-b918-1cdaae8cc539 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:18.342931 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89eb868-8e75-4832-b918-1cdaae8cc539 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:18.347965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae679c-e03a-46da-b24e-678d9c3e917f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:18.348582 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae679c-e03a-46da-b24e-678d9c3e917f admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:18.348892 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae679c-e03a-46da-b24e-678d9c3e917f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:18.349160 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae679c-e03a-46da-b24e-678d9c3e917f admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:18.349859 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adae679c-e03a-46da-b24e-678d9c3e917f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:18.351513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adae679c-e03a-46da-b24e-678d9c3e917f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:18.362537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 991/1958] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:26:18.368004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137ee4c-2196-4390-9dec-951c8ab48f96 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:18.378183 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137ee4c-2196-4390-9dec-951c8ab48f96 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:18.383354 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e2e98d-15a6-4eac-9e69-a570d83e59ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:18.384006 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2e98d-15a6-4eac-9e69-a570d83e59ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:18.384362 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2e98d-15a6-4eac-9e69-a570d83e59ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:18.384932 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2e98d-15a6-4eac-9e69-a570d83e59ca admin admin] PATH_INFO: `/v3/projects/38c97af2e29d4acab5df4051f7974b78/users/c5b21ca72c7a432ba31df98ac2b641f2/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:18.400040 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e2e98d-15a6-4eac-9e69-a570d83e59ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=38c97af2e29d4acab5df4051f7974b78, user_id=c5b21ca72c7a432ba31df98ac2b641f2, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:18.402735 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e2e98d-15a6-4eac-9e69-a570d83e59ca admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:18.417120 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 968/1959] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:26:18 2022] PUT /identity/v3/projects/38c97af2e29d4acab5df4051f7974b78/users/c5b21ca72c7a432ba31df98ac2b641f2/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:26:18.429867 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf722b-13cb-448d-ad1f-6306291835b3 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:18.430417 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf722b-13cb-448d-ad1f-6306291835b3 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:18.430417 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf722b-13cb-448d-ad1f-6306291835b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:18.458974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-28cf722b-13cb-448d-ad1f-6306291835b3 None None] MFA Rules not processed for user `c5b21ca72c7a432ba31df98ac2b641f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:26:18.460848 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-28cf722b-13cb-448d-ad1f-6306291835b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:18.468906 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:26:18.531965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 992/1960] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:26:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:26:18.543815 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e2e98d-15a6-4eac-9e69-a570d83e59ca admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:18.545732 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-23e2e98d-15a6-4eac-9e69-a570d83e59ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:18.642019 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e2e98d-15a6-4eac-9e69-a570d83e59ca admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:18.647560 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf8cbf-cc05-4497-a3ea-a8b581f8e65b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:18.648306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf8cbf-cc05-4497-a3ea-a8b581f8e65b service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:18.648581 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf8cbf-cc05-4497-a3ea-a8b581f8e65b service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:18.648877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf8cbf-cc05-4497-a3ea-a8b581f8e65b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:18.657058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf8cbf-cc05-4497-a3ea-a8b581f8e65b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:18.658666 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf8cbf-cc05-4497-a3ea-a8b581f8e65b service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:18.668613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 969/1961] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:26:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:26:28.528951 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=23 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:26:28.663894 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=67,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:26:36.372741 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c69fe-f1f7-46bd-96d5-2f838242499f None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:36.372741 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c69fe-f1f7-46bd-96d5-2f838242499f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:36.372741 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c69fe-f1f7-46bd-96d5-2f838242499f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:36.418426 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf8cbf-cc05-4497-a3ea-a8b581f8e65b service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:36.420659 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8baf8cbf-cc05-4497-a3ea-a8b581f8e65b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:36.448993 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-4d3c69fe-f1f7-46bd-96d5-2f838242499f None None] MFA Rules not processed for user `2c6be23065c643939055a3de375187d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:26:36.450356 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4d3c69fe-f1f7-46bd-96d5-2f838242499f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:36.536993 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf8cbf-cc05-4497-a3ea-a8b581f8e65b service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:36.543137 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863a79e-9666-459a-9745-ec397227004e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:36.544405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863a79e-9666-459a-9745-ec397227004e service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:36.545074 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863a79e-9666-459a-9745-ec397227004e service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:36.546039 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863a79e-9666-459a-9745-ec397227004e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:36.548865 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f863a79e-9666-459a-9745-ec397227004e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:36.550351 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 993/1962] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:26:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:26:36.666095 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863a79e-9666-459a-9745-ec397227004e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:36.667947 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863a79e-9666-459a-9745-ec397227004e service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:36.689987 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 970/1963] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:26:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:26:36.934147 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c69fe-f1f7-46bd-96d5-2f838242499f None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:36.935901 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4d3c69fe-f1f7-46bd-96d5-2f838242499f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:36.949244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c69fe-f1f7-46bd-96d5-2f838242499f None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:36.952938 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6231d0c8-ec90-44a3-9392-fe8e462ac4f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:36.953750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231d0c8-ec90-44a3-9392-fe8e462ac4f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:36.954034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231d0c8-ec90-44a3-9392-fe8e462ac4f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:36.954306 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231d0c8-ec90-44a3-9392-fe8e462ac4f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:36.956143 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6231d0c8-ec90-44a3-9392-fe8e462ac4f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:36.967508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6231d0c8-ec90-44a3-9392-fe8e462ac4f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:36.968980 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6231d0c8-ec90-44a3-9392-fe8e462ac4f3 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:36.977813 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 994/1964] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:26:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:26:37.350489 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863a79e-9666-459a-9745-ec397227004e service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:37.353102 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f863a79e-9666-459a-9745-ec397227004e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:37.456036 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863a79e-9666-459a-9745-ec397227004e service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:37.461473 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679c32c-5cc0-4fd0-a462-00a63ef8f941 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfbaaf32475a4c82a34948602d8f4e35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:37.462137 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679c32c-5cc0-4fd0-a462-00a63ef8f941 service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:37.462410 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679c32c-5cc0-4fd0-a462-00a63ef8f941 service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:37.462676 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679c32c-5cc0-4fd0-a462-00a63ef8f941 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:37.464366 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f679c32c-5cc0-4fd0-a462-00a63ef8f941 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:37.476774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f679c32c-5cc0-4fd0-a462-00a63ef8f941 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:37.478364 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f679c32c-5cc0-4fd0-a462-00a63ef8f941 service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:37.488747 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 971/1965] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:26:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:26:45.144293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c8864-03fc-44f9-a57e-72674878a815 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:45.144859 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c8864-03fc-44f9-a57e-72674878a815 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:45.145371 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c8864-03fc-44f9-a57e-72674878a815 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:45.147144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 995/1966] 10.222.0.20 () {54 vars in 921 bytes} [Thu Oct 20 06:26:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 06:26:45.165136 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501251ba-5acb-4c3e-aafe-238fdba4d688 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:45.166235 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501251ba-5acb-4c3e-aafe-238fdba4d688 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:45.166737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501251ba-5acb-4c3e-aafe-238fdba4d688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:45.244872 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:26:45.246304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c8864-03fc-44f9-a57e-72674878a815 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:45.248902 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-501251ba-5acb-4c3e-aafe-238fdba4d688 None None] MFA Rules not processed for user `cfbaaf32475a4c82a34948602d8f4e35`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:26:45.250190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-746c8864-03fc-44f9-a57e-72674878a815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:45.250977 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-501251ba-5acb-4c3e-aafe-238fdba4d688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:45.262565 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 972/1967] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Oct 20 06:26:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:26:45.269428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c8864-03fc-44f9-a57e-72674878a815 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:45.279736 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501251ba-5acb-4c3e-aafe-238fdba4d688 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:45.282601 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97d30d-e4e3-4671-b586-88d8161a057b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:45.283415 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97d30d-e4e3-4671-b586-88d8161a057b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:45.283415 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97d30d-e4e3-4671-b586-88d8161a057b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:45.283643 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-501251ba-5acb-4c3e-aafe-238fdba4d688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:45.283737 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97d30d-e4e3-4671-b586-88d8161a057b admin admin] PATH_INFO: `/v3/users/b2a67495335541dcb1cf1405ca9e3245` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:45.284942 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa97d30d-e4e3-4671-b586-88d8161a057b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2a67495335541dcb1cf1405ca9e3245)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:45.287219 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa97d30d-e4e3-4671-b586-88d8161a057b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:45.307442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501251ba-5acb-4c3e-aafe-238fdba4d688 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:45.314769 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad973f9-eef0-445a-8d74-31cebaf7a830 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfbaaf32475a4c82a34948602d8f4e35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:45.315912 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad973f9-eef0-445a-8d74-31cebaf7a830 service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:45.316302 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad973f9-eef0-445a-8d74-31cebaf7a830 service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:45.316719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad973f9-eef0-445a-8d74-31cebaf7a830 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:45.329416 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad973f9-eef0-445a-8d74-31cebaf7a830 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:45.331607 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad973f9-eef0-445a-8d74-31cebaf7a830 service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:45.352306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 973/1968] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:26:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:26:45.361529 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-aa97d30d-e4e3-4671-b586-88d8161a057b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2a67495335541dcb1cf1405ca9e3245'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:45.362415 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-aa97d30d-e4e3-4671-b586-88d8161a057b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2a67495335541dcb1cf1405ca9e3245'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:45.374367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-aa97d30d-e4e3-4671-b586-88d8161a057b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2a67495335541dcb1cf1405ca9e3245'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:45.392028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-aa97d30d-e4e3-4671-b586-88d8161a057b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2a67495335541dcb1cf1405ca9e3245'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:45.392871 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-aa97d30d-e4e3-4671-b586-88d8161a057b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2a67495335541dcb1cf1405ca9e3245'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:45.418888 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 996/1969] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:26:45 2022] DELETE /identity/v3/users/b2a67495335541dcb1cf1405ca9e3245 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:26:45.679226 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad973f9-eef0-445a-8d74-31cebaf7a830 service placement] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:45.690884 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad973f9-eef0-445a-8d74-31cebaf7a830 service placement] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:45.768586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357cd803-6aa9-402f-8b15-baa315c7787d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:45.769260 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357cd803-6aa9-402f-8b15-baa315c7787d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:45.769915 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357cd803-6aa9-402f-8b15-baa315c7787d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:45.770194 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357cd803-6aa9-402f-8b15-baa315c7787d admin admin] PATH_INFO: `/v3/projects/0cfa9834ec244713816c8775c3ba7f18` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:45.771306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357cd803-6aa9-402f-8b15-baa315c7787d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cfa9834ec244713816c8775c3ba7f18)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:45.773358 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357cd803-6aa9-402f-8b15-baa315c7787d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:45.819638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-357cd803-6aa9-402f-8b15-baa315c7787d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cfa9834ec244713816c8775c3ba7f18', 'request_id': 'req-357cd803-6aa9-402f-8b15-baa315c7787d'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:45.825248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-357cd803-6aa9-402f-8b15-baa315c7787d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cfa9834ec244713816c8775c3ba7f18', 'request_id': 'req-357cd803-6aa9-402f-8b15-baa315c7787d'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:45.831711 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-357cd803-6aa9-402f-8b15-baa315c7787d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cfa9834ec244713816c8775c3ba7f18', 'request_id': 'req-357cd803-6aa9-402f-8b15-baa315c7787d'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:45.850048 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-357cd803-6aa9-402f-8b15-baa315c7787d admin admin] The token cache is being invalidate because project 0cfa9834ec244713816c8775c3ba7f18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:26:45.850340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-357cd803-6aa9-402f-8b15-baa315c7787d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:45.851688 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-357cd803-6aa9-402f-8b15-baa315c7787d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:45.853387 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 974/1970] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:26:45 2022] DELETE /identity/v3/projects/0cfa9834ec244713816c8775c3ba7f18 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:26:45.857634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97d30d-e4e3-4671-b586-88d8161a057b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:45.859221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-aa97d30d-e4e3-4671-b586-88d8161a057b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:45.872929 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97d30d-e4e3-4671-b586-88d8161a057b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:45.901294 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:26:45.935796 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a00087-ffc1-40f9-939a-b9d6cea0ccb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:45.936430 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a00087-ffc1-40f9-939a-b9d6cea0ccb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:45.936749 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a00087-ffc1-40f9-939a-b9d6cea0ccb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:45.937020 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a00087-ffc1-40f9-939a-b9d6cea0ccb5 admin admin] PATH_INFO: `/v3/users/639e084a25f04167afff8739730e6d71` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:45.938324 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a00087-ffc1-40f9-939a-b9d6cea0ccb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=639e084a25f04167afff8739730e6d71)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:45.940266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a00087-ffc1-40f9-939a-b9d6cea0ccb5 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:45.988626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a7a00087-ffc1-40f9-939a-b9d6cea0ccb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '639e084a25f04167afff8739730e6d71'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:45.989289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a7a00087-ffc1-40f9-939a-b9d6cea0ccb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '639e084a25f04167afff8739730e6d71'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:45.998125 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a7a00087-ffc1-40f9-939a-b9d6cea0ccb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '639e084a25f04167afff8739730e6d71'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:46.011167 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a7a00087-ffc1-40f9-939a-b9d6cea0ccb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '639e084a25f04167afff8739730e6d71'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:46.011820 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a7a00087-ffc1-40f9-939a-b9d6cea0ccb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '639e084a25f04167afff8739730e6d71'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:46.023934 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 997/1971] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:26:45 2022] DELETE /identity/v3/users/639e084a25f04167afff8739730e6d71 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:26:46.203348 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357cd803-6aa9-402f-8b15-baa315c7787d admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:46.205478 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-357cd803-6aa9-402f-8b15-baa315c7787d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:46.220743 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357cd803-6aa9-402f-8b15-baa315c7787d admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:46.284559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87349db-65b7-42d4-8935-cc9dd86233fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:46.285165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87349db-65b7-42d4-8935-cc9dd86233fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:46.285463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87349db-65b7-42d4-8935-cc9dd86233fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:46.285738 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87349db-65b7-42d4-8935-cc9dd86233fb admin admin] PATH_INFO: `/v3/projects/47e2a0ec4763475fa1a047220baea2c7` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:46.286857 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87349db-65b7-42d4-8935-cc9dd86233fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=47e2a0ec4763475fa1a047220baea2c7)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:46.290262 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87349db-65b7-42d4-8935-cc9dd86233fb admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:46.318845 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:26:46.345330 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d87349db-65b7-42d4-8935-cc9dd86233fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47e2a0ec4763475fa1a047220baea2c7', 'request_id': 'req-d87349db-65b7-42d4-8935-cc9dd86233fb'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:46.351522 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d87349db-65b7-42d4-8935-cc9dd86233fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47e2a0ec4763475fa1a047220baea2c7', 'request_id': 'req-d87349db-65b7-42d4-8935-cc9dd86233fb'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:46.360283 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d87349db-65b7-42d4-8935-cc9dd86233fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47e2a0ec4763475fa1a047220baea2c7', 'request_id': 'req-d87349db-65b7-42d4-8935-cc9dd86233fb'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:46.373824 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d87349db-65b7-42d4-8935-cc9dd86233fb admin admin] The token cache is being invalidate because project 47e2a0ec4763475fa1a047220baea2c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:26:46.374114 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d87349db-65b7-42d4-8935-cc9dd86233fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:46.374837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d87349db-65b7-42d4-8935-cc9dd86233fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:26:46.376436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 975/1972] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:26:46 2022] DELETE /identity/v3/projects/47e2a0ec4763475fa1a047220baea2c7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:26:46.381283 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61bb79-76e9-4483-8ffa-3c5659fd232f None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:46.382339 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61bb79-76e9-4483-8ffa-3c5659fd232f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:46.382339 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61bb79-76e9-4483-8ffa-3c5659fd232f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:46.403161 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-bc61bb79-76e9-4483-8ffa-3c5659fd232f None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:26:46.404356 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-bc61bb79-76e9-4483-8ffa-3c5659fd232f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:46.464657 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 998/1973] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:26:46.473821 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062ed5b-2204-4da4-8891-21cc5893d3bd None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:46.474101 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062ed5b-2204-4da4-8891-21cc5893d3bd None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:46.474375 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062ed5b-2204-4da4-8891-21cc5893d3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:46.495458 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f062ed5b-2204-4da4-8891-21cc5893d3bd None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:26:46.496634 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f062ed5b-2204-4da4-8891-21cc5893d3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:46.504426 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 976/1974] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:26:46.508750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61bb79-76e9-4483-8ffa-3c5659fd232f None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:46.520206 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61bb79-76e9-4483-8ffa-3c5659fd232f None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:46.525461 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf50a0f4-5d21-4b3e-abf7-0ea868bd7906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:46.526058 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50a0f4-5d21-4b3e-abf7-0ea868bd7906 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:46.526333 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50a0f4-5d21-4b3e-abf7-0ea868bd7906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:46.526598 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50a0f4-5d21-4b3e-abf7-0ea868bd7906 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:46.527232 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf50a0f4-5d21-4b3e-abf7-0ea868bd7906 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:26:46.527526 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf50a0f4-5d21-4b3e-abf7-0ea868bd7906 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:46.529138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf50a0f4-5d21-4b3e-abf7-0ea868bd7906 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:46.541487 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 999/1975] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:26:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:26:46.545809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062ed5b-2204-4da4-8891-21cc5893d3bd None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:46.555723 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062ed5b-2204-4da4-8891-21cc5893d3bd None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:46.560602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eec43a-e6a2-41e8-8d66-ca68bb46ed16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:46.561298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eec43a-e6a2-41e8-8d66-ca68bb46ed16 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:46.561584 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eec43a-e6a2-41e8-8d66-ca68bb46ed16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:46.561859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eec43a-e6a2-41e8-8d66-ca68bb46ed16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:46.562510 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eec43a-e6a2-41e8-8d66-ca68bb46ed16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:46.564419 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eec43a-e6a2-41e8-8d66-ca68bb46ed16 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:46.600467 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 977/1976] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:26:46 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:26:46.601961 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf50a0f4-5d21-4b3e-abf7-0ea868bd7906 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:46.612513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf50a0f4-5d21-4b3e-abf7-0ea868bd7906 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:46.648423 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:26:46.684217 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33675c5e-a7c2-4aa4-8d19-4bce1a4e2315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:46.684866 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33675c5e-a7c2-4aa4-8d19-4bce1a4e2315 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:46.685190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33675c5e-a7c2-4aa4-8d19-4bce1a4e2315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:46.685465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33675c5e-a7c2-4aa4-8d19-4bce1a4e2315 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:46.686089 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33675c5e-a7c2-4aa4-8d19-4bce1a4e2315 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:46.688271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33675c5e-a7c2-4aa4-8d19-4bce1a4e2315 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:46.712719 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1000/1977] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:26:46 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:26:46.716807 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eec43a-e6a2-41e8-8d66-ca68bb46ed16 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:46.727112 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eec43a-e6a2-41e8-8d66-ca68bb46ed16 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:46.735261 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8432d6-a5bd-4c9c-beb7-c6bc34906289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:46.736122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8432d6-a5bd-4c9c-beb7-c6bc34906289 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:46.736370 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8432d6-a5bd-4c9c-beb7-c6bc34906289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:46.736677 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8432d6-a5bd-4c9c-beb7-c6bc34906289 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:46.737393 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8432d6-a5bd-4c9c-beb7-c6bc34906289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:46.739122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8432d6-a5bd-4c9c-beb7-c6bc34906289 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:46.756096 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 978/1978] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:26:46.760947 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33675c5e-a7c2-4aa4-8d19-4bce1a4e2315 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:46.770700 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33675c5e-a7c2-4aa4-8d19-4bce1a4e2315 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:46.775437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc4955-2375-4629-8d1e-3bd2fc20baeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:46.776033 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc4955-2375-4629-8d1e-3bd2fc20baeb admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:46.776309 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc4955-2375-4629-8d1e-3bd2fc20baeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:46.776577 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc4955-2375-4629-8d1e-3bd2fc20baeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:46.779983 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc4955-2375-4629-8d1e-3bd2fc20baeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:46.779983 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc4955-2375-4629-8d1e-3bd2fc20baeb admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:46.799995 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1001/1979] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:26:46.804584 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8432d6-a5bd-4c9c-beb7-c6bc34906289 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:46.819040 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8432d6-a5bd-4c9c-beb7-c6bc34906289 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:46.825031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61701cc1-f163-416d-89cb-7b772bb638d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:46.825711 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61701cc1-f163-416d-89cb-7b772bb638d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:46.825989 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61701cc1-f163-416d-89cb-7b772bb638d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:46.826263 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61701cc1-f163-416d-89cb-7b772bb638d6 admin admin] PATH_INFO: `/v3/projects/db144186ebfa49c185816e5c986b6098/users/34992b551ea54a1fbe128e287e014847/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:46.842324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61701cc1-f163-416d-89cb-7b772bb638d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db144186ebfa49c185816e5c986b6098, user_id=34992b551ea54a1fbe128e287e014847, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:46.845149 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61701cc1-f163-416d-89cb-7b772bb638d6 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:46.858790 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 979/1980] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:26:46 2022] PUT /identity/v3/projects/db144186ebfa49c185816e5c986b6098/users/34992b551ea54a1fbe128e287e014847/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:26:46.864009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0fe7a-cc35-41b0-9f9a-652b53861841 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:46.864344 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0fe7a-cc35-41b0-9f9a-652b53861841 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:46.864624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0fe7a-cc35-41b0-9f9a-652b53861841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:46.883898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-70d0fe7a-cc35-41b0-9f9a-652b53861841 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:26:46.885367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-70d0fe7a-cc35-41b0-9f9a-652b53861841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:46.942803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1002/1981] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:26:46.953181 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61701cc1-f163-416d-89cb-7b772bb638d6 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:46.954911 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-61701cc1-f163-416d-89cb-7b772bb638d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:47.022604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61701cc1-f163-416d-89cb-7b772bb638d6 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:47.026457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd80e04-81be-4783-ba0e-5a7e848f2aa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:47.027073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd80e04-81be-4783-ba0e-5a7e848f2aa3 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:47.027355 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd80e04-81be-4783-ba0e-5a7e848f2aa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:47.027631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd80e04-81be-4783-ba0e-5a7e848f2aa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:47.037023 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd80e04-81be-4783-ba0e-5a7e848f2aa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:47.039135 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd80e04-81be-4783-ba0e-5a7e848f2aa3 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:47.058904 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 980/1982] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:26:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:26:47.140586 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0fe7a-cc35-41b0-9f9a-652b53861841 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:47.142294 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-70d0fe7a-cc35-41b0-9f9a-652b53861841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:47.318422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0fe7a-cc35-41b0-9f9a-652b53861841 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:47.324940 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab5606-08a7-4baf-8c6a-bdcbdfd7a19b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:47.325551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab5606-08a7-4baf-8c6a-bdcbdfd7a19b service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:47.325831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab5606-08a7-4baf-8c6a-bdcbdfd7a19b service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:47.326260 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab5606-08a7-4baf-8c6a-bdcbdfd7a19b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:47.328033 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-dbab5606-08a7-4baf-8c6a-bdcbdfd7a19b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:47.342392 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab5606-08a7-4baf-8c6a-bdcbdfd7a19b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:47.344624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab5606-08a7-4baf-8c6a-bdcbdfd7a19b service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:47.356575 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1003/1983] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:26:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:26:52.384404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928eb61d-23ea-4919-8e67-2afde557acaf None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:52.385252 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928eb61d-23ea-4919-8e67-2afde557acaf None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:52.385252 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928eb61d-23ea-4919-8e67-2afde557acaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:52.424954 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-928eb61d-23ea-4919-8e67-2afde557acaf None None] MFA Rules not processed for user `34992b551ea54a1fbe128e287e014847`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:26:52.426512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-928eb61d-23ea-4919-8e67-2afde557acaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:52.488953 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:26:52.557618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 981/1984] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:26:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:26:52.563876 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab5606-08a7-4baf-8c6a-bdcbdfd7a19b service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:52.574617 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab5606-08a7-4baf-8c6a-bdcbdfd7a19b service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:52.579309 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740df7c-554e-4458-a986-c22318a46d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:52.580006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740df7c-554e-4458-a986-c22318a46d18 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:52.580288 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740df7c-554e-4458-a986-c22318a46d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:52.580748 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740df7c-554e-4458-a986-c22318a46d18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:52.581365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9740df7c-554e-4458-a986-c22318a46d18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:52.583449 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9740df7c-554e-4458-a986-c22318a46d18 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:52.626225 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1004/1985] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:26:52 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:26:52.628063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928eb61d-23ea-4919-8e67-2afde557acaf None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:52.647992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928eb61d-23ea-4919-8e67-2afde557acaf None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:52.769277 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b6dc9-f2c7-4ed3-8476-0b2d0f98bfe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:52.770063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b6dc9-f2c7-4ed3-8476-0b2d0f98bfe7 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:52.770949 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b6dc9-f2c7-4ed3-8476-0b2d0f98bfe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:52.771333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b6dc9-f2c7-4ed3-8476-0b2d0f98bfe7 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:52.773127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b6dc9-f2c7-4ed3-8476-0b2d0f98bfe7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:52.781972 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b6dc9-f2c7-4ed3-8476-0b2d0f98bfe7 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:52.807282 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740df7c-554e-4458-a986-c22318a46d18 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:52.832231 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 982/1986] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:26:52 2022] POST /identity/v3/users => generated 296 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:26:52.833699 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b6dc9-f2c7-4ed3-8476-0b2d0f98bfe7 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:52.840974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:26:52.857789 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b6dc9-f2c7-4ed3-8476-0b2d0f98bfe7 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:52.867799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce49ef-8f96-44cd-ab2e-c6c017c658ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:52.869222 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce49ef-8f96-44cd-ab2e-c6c017c658ac admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:52.869709 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce49ef-8f96-44cd-ab2e-c6c017c658ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:52.871089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce49ef-8f96-44cd-ab2e-c6c017c658ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:52.872041 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce49ef-8f96-44cd-ab2e-c6c017c658ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:52.879335 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce49ef-8f96-44cd-ab2e-c6c017c658ac admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:52.896666 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 983/1987] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:26:52.900763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce49ef-8f96-44cd-ab2e-c6c017c658ac admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:52.915385 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce49ef-8f96-44cd-ab2e-c6c017c658ac admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:52.921134 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45fc8a-5268-451e-bfdc-2ce42f62e567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:52.922070 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45fc8a-5268-451e-bfdc-2ce42f62e567 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:52.922453 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45fc8a-5268-451e-bfdc-2ce42f62e567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:52.922826 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45fc8a-5268-451e-bfdc-2ce42f62e567 admin admin] PATH_INFO: `/v3/projects/08da307ec97b476f89e0be61ef9a6292/users/03c353d72f4c4bde896ba11287614d1e/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:52.939343 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740df7c-554e-4458-a986-c22318a46d18 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:52.942392 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45fc8a-5268-451e-bfdc-2ce42f62e567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08da307ec97b476f89e0be61ef9a6292, user_id=03c353d72f4c4bde896ba11287614d1e, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:52.946941 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45fc8a-5268-451e-bfdc-2ce42f62e567 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:52.955812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e099d4-fc56-4f2e-9b67-375682de5987 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:52.957008 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e099d4-fc56-4f2e-9b67-375682de5987 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:52.957495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e099d4-fc56-4f2e-9b67-375682de5987 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:52.958043 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e099d4-fc56-4f2e-9b67-375682de5987 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:52.961027 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-18e099d4-fc56-4f2e-9b67-375682de5987 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:52.967839 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 984/1988] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:26:52 2022] PUT /identity/v3/projects/08da307ec97b476f89e0be61ef9a6292/users/03c353d72f4c4bde896ba11287614d1e/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:26:52.972157 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45fc8a-5268-451e-bfdc-2ce42f62e567 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:52.979815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e099d4-fc56-4f2e-9b67-375682de5987 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:52.990585 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e099d4-fc56-4f2e-9b67-375682de5987 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:53.023782 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1005/1989] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:26:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:26:53.055311 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45fc8a-5268-451e-bfdc-2ce42f62e567 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:53.061615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8ac77-8e7c-4b0a-894d-50462b8c56d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:53.062516 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8ac77-8e7c-4b0a-894d-50462b8c56d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:53.062881 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8ac77-8e7c-4b0a-894d-50462b8c56d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:53.063226 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8ac77-8e7c-4b0a-894d-50462b8c56d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:53.064120 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8ac77-8e7c-4b0a-894d-50462b8c56d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:53.066746 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8ac77-8e7c-4b0a-894d-50462b8c56d7 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:53.088558 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 985/1990] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:26:53.093035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e099d4-fc56-4f2e-9b67-375682de5987 service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:53.107542 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e099d4-fc56-4f2e-9b67-375682de5987 service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:53.116479 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb6b33-abad-4272-b16a-4b8b47b1f1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:53.117296 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb6b33-abad-4272-b16a-4b8b47b1f1a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:53.117622 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb6b33-abad-4272-b16a-4b8b47b1f1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:53.117899 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb6b33-abad-4272-b16a-4b8b47b1f1a7 admin admin] PATH_INFO: `/v3/projects/08da307ec97b476f89e0be61ef9a6292/users/03c353d72f4c4bde896ba11287614d1e/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:53.119867 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb6b33-abad-4272-b16a-4b8b47b1f1a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08da307ec97b476f89e0be61ef9a6292, user_id=03c353d72f4c4bde896ba11287614d1e, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:53.123359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb6b33-abad-4272-b16a-4b8b47b1f1a7 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:53.142415 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1006/1991] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:26:53 2022] PUT /identity/v3/projects/08da307ec97b476f89e0be61ef9a6292/users/03c353d72f4c4bde896ba11287614d1e/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:26:59.089044 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62e82b-5931-4b4d-88b2-0696f5ebfff3 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:59.089532 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62e82b-5931-4b4d-88b2-0696f5ebfff3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:59.089702 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62e82b-5931-4b4d-88b2-0696f5ebfff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:59.153975 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-1c62e82b-5931-4b4d-88b2-0696f5ebfff3 None None] MFA Rules not processed for user `03c353d72f4c4bde896ba11287614d1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:26:59.155190 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1c62e82b-5931-4b4d-88b2-0696f5ebfff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:26:59.174920 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:26:59.254373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 986/1992] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:26:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:26:59.271389 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb6b33-abad-4272-b16a-4b8b47b1f1a7 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:26:59.365252 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb6b33-abad-4272-b16a-4b8b47b1f1a7 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:26:59.372485 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8730a-4bff-459d-b3ef-50584d85b89f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:26:59.373134 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8730a-4bff-459d-b3ef-50584d85b89f service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:26:59.373419 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8730a-4bff-459d-b3ef-50584d85b89f service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:26:59.373690 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8730a-4bff-459d-b3ef-50584d85b89f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:26:59.441310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8730a-4bff-459d-b3ef-50584d85b89f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:26:59.442919 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8730a-4bff-459d-b3ef-50584d85b89f service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:26:59.454331 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1007/1993] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:26:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:27:09.251140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=22 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:27:09.449516 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=87,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:27:18.613745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62e82b-5931-4b4d-88b2-0696f5ebfff3 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:27:18.615129 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1c62e82b-5931-4b4d-88b2-0696f5ebfff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:27:18.682051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62e82b-5931-4b4d-88b2-0696f5ebfff3 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:27:18.686628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db840c61-54fa-4072-8897-c88c349e0570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:27:18.687266 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db840c61-54fa-4072-8897-c88c349e0570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:27:18.687678 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db840c61-54fa-4072-8897-c88c349e0570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:27:18.688003 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db840c61-54fa-4072-8897-c88c349e0570 admin admin] PATH_INFO: `/v3/users/c5b21ca72c7a432ba31df98ac2b641f2` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:27:18.689173 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db840c61-54fa-4072-8897-c88c349e0570 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5b21ca72c7a432ba31df98ac2b641f2)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:27:18.691089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db840c61-54fa-4072-8897-c88c349e0570 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:27:18.738869 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-db840c61-54fa-4072-8897-c88c349e0570 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5b21ca72c7a432ba31df98ac2b641f2'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:27:18.739514 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-db840c61-54fa-4072-8897-c88c349e0570 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5b21ca72c7a432ba31df98ac2b641f2'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:27:18.747178 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-db840c61-54fa-4072-8897-c88c349e0570 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5b21ca72c7a432ba31df98ac2b641f2'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:27:18.761247 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-db840c61-54fa-4072-8897-c88c349e0570 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5b21ca72c7a432ba31df98ac2b641f2'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:27:18.768177 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-db840c61-54fa-4072-8897-c88c349e0570 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5b21ca72c7a432ba31df98ac2b641f2'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:27:18.774154 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 987/1994] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:27:18 2022] DELETE /identity/v3/users/c5b21ca72c7a432ba31df98ac2b641f2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:27:18.778695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b807bc5-bfdd-4d9f-9e9d-4c948429ebbf None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:27:18.779163 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b807bc5-bfdd-4d9f-9e9d-4c948429ebbf None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:27:18.779284 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b807bc5-bfdd-4d9f-9e9d-4c948429ebbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:27:18.799390 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-6b807bc5-bfdd-4d9f-9e9d-4c948429ebbf None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:27:18.800577 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6b807bc5-bfdd-4d9f-9e9d-4c948429ebbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:27:18.859619 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1008/1995] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:27:18.870678 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db840c61-54fa-4072-8897-c88c349e0570 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:27:18.872014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-db840c61-54fa-4072-8897-c88c349e0570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:27:18.929676 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db840c61-54fa-4072-8897-c88c349e0570 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:27:18.933997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef5076-7c5a-4a9f-86c7-f00bcbd54c83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:27:18.934596 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef5076-7c5a-4a9f-86c7-f00bcbd54c83 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:27:18.934873 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef5076-7c5a-4a9f-86c7-f00bcbd54c83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:27:18.935143 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef5076-7c5a-4a9f-86c7-f00bcbd54c83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:27:18.944071 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef5076-7c5a-4a9f-86c7-f00bcbd54c83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:27:18.945513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef5076-7c5a-4a9f-86c7-f00bcbd54c83 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:27:18.955505 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 988/1996] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:27:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:27:19.234868 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b807bc5-bfdd-4d9f-9e9d-4c948429ebbf None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:27:19.236796 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6b807bc5-bfdd-4d9f-9e9d-4c948429ebbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:27:19.253487 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b807bc5-bfdd-4d9f-9e9d-4c948429ebbf None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:27:19.260236 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c8c45-bf3e-48e8-a940-04ed89163c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:27:19.261367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c8c45-bf3e-48e8-a940-04ed89163c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:27:19.262075 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c8c45-bf3e-48e8-a940-04ed89163c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:27:19.262429 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c8c45-bf3e-48e8-a940-04ed89163c38 admin admin] PATH_INFO: `/v3/projects/38c97af2e29d4acab5df4051f7974b78` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:27:19.263705 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c8c45-bf3e-48e8-a940-04ed89163c38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38c97af2e29d4acab5df4051f7974b78)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:27:19.268360 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c8c45-bf3e-48e8-a940-04ed89163c38 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:27:19.325760 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-945c8c45-bf3e-48e8-a940-04ed89163c38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38c97af2e29d4acab5df4051f7974b78', 'request_id': 'req-945c8c45-bf3e-48e8-a940-04ed89163c38'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:27:19.331607 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-945c8c45-bf3e-48e8-a940-04ed89163c38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38c97af2e29d4acab5df4051f7974b78', 'request_id': 'req-945c8c45-bf3e-48e8-a940-04ed89163c38'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:27:19.338680 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-945c8c45-bf3e-48e8-a940-04ed89163c38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38c97af2e29d4acab5df4051f7974b78', 'request_id': 'req-945c8c45-bf3e-48e8-a940-04ed89163c38'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:27:19.353247 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-945c8c45-bf3e-48e8-a940-04ed89163c38 admin admin] The token cache is being invalidate because project 38c97af2e29d4acab5df4051f7974b78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:27:19.353489 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-945c8c45-bf3e-48e8-a940-04ed89163c38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:27:19.354019 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-945c8c45-bf3e-48e8-a940-04ed89163c38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:27:19.355505 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1009/1997] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:27:19 2022] DELETE /identity/v3/projects/38c97af2e29d4acab5df4051f7974b78 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:27:19.360469 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e55ac-f901-4b4d-88a7-0404f50909e1 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:27:19.360817 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e55ac-f901-4b4d-88a7-0404f50909e1 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:27:19.361093 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e55ac-f901-4b4d-88a7-0404f50909e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:27:19.381205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-e05e55ac-f901-4b4d-88a7-0404f50909e1 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:27:19.382481 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e05e55ac-f901-4b4d-88a7-0404f50909e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:27:19.400984 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:27:19.446863 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 989/1998] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:27:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:27:19.454852 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33839b-2866-49c0-951f-3d16cf3ca637 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:27:19.455135 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33839b-2866-49c0-951f-3d16cf3ca637 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:27:19.455417 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33839b-2866-49c0-951f-3d16cf3ca637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:27:19.475547 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-1f33839b-2866-49c0-951f-3d16cf3ca637 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:27:19.476744 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1f33839b-2866-49c0-951f-3d16cf3ca637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:27:19.485250 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1010/1999] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:27:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:27:19.489479 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e55ac-f901-4b4d-88a7-0404f50909e1 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:27:19.499330 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e55ac-f901-4b4d-88a7-0404f50909e1 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:27:19.503821 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4bc8c-4455-4189-a3b0-29ab83cc0572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:27:19.504424 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4bc8c-4455-4189-a3b0-29ab83cc0572 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:27:19.504717 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4bc8c-4455-4189-a3b0-29ab83cc0572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:27:19.505005 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4bc8c-4455-4189-a3b0-29ab83cc0572 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:27:19.505638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4bc8c-4455-4189-a3b0-29ab83cc0572 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:27:19.505935 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4bc8c-4455-4189-a3b0-29ab83cc0572 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:27:19.507510 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4bc8c-4455-4189-a3b0-29ab83cc0572 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:27:19.519469 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 990/2000] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:27:19.523559 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33839b-2866-49c0-951f-3d16cf3ca637 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:27:19.534077 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33839b-2866-49c0-951f-3d16cf3ca637 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:27:19.538689 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e47705-5f9e-45fc-9d06-48179d3ed64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:27:19.539285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e47705-5f9e-45fc-9d06-48179d3ed64b admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:27:19.540176 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e47705-5f9e-45fc-9d06-48179d3ed64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:27:19.540449 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e47705-5f9e-45fc-9d06-48179d3ed64b admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:27:19.541246 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e47705-5f9e-45fc-9d06-48179d3ed64b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:27:19.543128 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e47705-5f9e-45fc-9d06-48179d3ed64b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:27:19.570077 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1011/2001] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:27:19 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:27:19.574264 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4bc8c-4455-4189-a3b0-29ab83cc0572 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:27:19.584502 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4bc8c-4455-4189-a3b0-29ab83cc0572 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:27:19.641227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31a7fe-5bf8-4a1d-8e1d-c6f5da1832be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:27:19.641973 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31a7fe-5bf8-4a1d-8e1d-c6f5da1832be admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:27:19.642255 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31a7fe-5bf8-4a1d-8e1d-c6f5da1832be admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:27:19.642524 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31a7fe-5bf8-4a1d-8e1d-c6f5da1832be admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:27:19.643155 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31a7fe-5bf8-4a1d-8e1d-c6f5da1832be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:27:19.645113 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31a7fe-5bf8-4a1d-8e1d-c6f5da1832be admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:27:19.670852 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 991/2002] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:27:19 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:27:19.672097 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e47705-5f9e-45fc-9d06-48179d3ed64b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:27:19.683662 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e47705-5f9e-45fc-9d06-48179d3ed64b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:27:19.688413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef15a8-fc01-4b56-b2ee-9ca2ebd2df9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:27:19.689051 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef15a8-fc01-4b56-b2ee-9ca2ebd2df9a admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:27:19.689346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef15a8-fc01-4b56-b2ee-9ca2ebd2df9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:27:19.689615 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef15a8-fc01-4b56-b2ee-9ca2ebd2df9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:27:19.690244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef15a8-fc01-4b56-b2ee-9ca2ebd2df9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:27:19.691822 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef15a8-fc01-4b56-b2ee-9ca2ebd2df9a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:27:19.705253 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1012/2003] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:27:19.706227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31a7fe-5bf8-4a1d-8e1d-c6f5da1832be admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:27:19.718143 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31a7fe-5bf8-4a1d-8e1d-c6f5da1832be admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:27:19.723643 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54095f63-ea1e-408b-9f8c-216402ed33e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:27:19.724256 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54095f63-ea1e-408b-9f8c-216402ed33e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:27:19.724567 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54095f63-ea1e-408b-9f8c-216402ed33e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:27:19.724878 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54095f63-ea1e-408b-9f8c-216402ed33e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:27:19.725548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54095f63-ea1e-408b-9f8c-216402ed33e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:27:19.727719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54095f63-ea1e-408b-9f8c-216402ed33e2 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:27:19.738861 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 992/2004] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:27:19.744389 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef15a8-fc01-4b56-b2ee-9ca2ebd2df9a admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:27:19.754471 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef15a8-fc01-4b56-b2ee-9ca2ebd2df9a admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:27:19.759145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62f636-01f6-4304-8e5a-c86789f6b259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:27:19.760046 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62f636-01f6-4304-8e5a-c86789f6b259 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:27:19.760321 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62f636-01f6-4304-8e5a-c86789f6b259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:27:19.760592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62f636-01f6-4304-8e5a-c86789f6b259 admin admin] PATH_INFO: `/v3/projects/8fdac492af4040c4957ca72f0849657f/users/ec0cf867400c4ec3b880f5258ed9b0fd/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:27:19.775790 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62f636-01f6-4304-8e5a-c86789f6b259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fdac492af4040c4957ca72f0849657f, user_id=ec0cf867400c4ec3b880f5258ed9b0fd, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:27:19.778442 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62f636-01f6-4304-8e5a-c86789f6b259 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:27:19.794303 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1013/2005] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:27:19 2022] PUT /identity/v3/projects/8fdac492af4040c4957ca72f0849657f/users/ec0cf867400c4ec3b880f5258ed9b0fd/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:27:19.805152 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff16f39-55e8-411d-a364-7f0e9d9f839e None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:27:19.805583 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff16f39-55e8-411d-a364-7f0e9d9f839e None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:27:19.805746 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff16f39-55e8-411d-a364-7f0e9d9f839e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:27:19.835711 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-6ff16f39-55e8-411d-a364-7f0e9d9f839e None None] MFA Rules not processed for user `ec0cf867400c4ec3b880f5258ed9b0fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:27:19.837648 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6ff16f39-55e8-411d-a364-7f0e9d9f839e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:27:19.902178 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 993/2006] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:27:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:27:19.906822 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62f636-01f6-4304-8e5a-c86789f6b259 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:27:19.927874 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:27:19.971837 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62f636-01f6-4304-8e5a-c86789f6b259 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:27:19.976774 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169f9ee-5f1e-4782-a0fa-dec388c1b513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:27:19.978301 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169f9ee-5f1e-4782-a0fa-dec388c1b513 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:27:19.978588 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169f9ee-5f1e-4782-a0fa-dec388c1b513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:27:19.978862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169f9ee-5f1e-4782-a0fa-dec388c1b513 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:27:19.979503 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a169f9ee-5f1e-4782-a0fa-dec388c1b513 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:27:19.981425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a169f9ee-5f1e-4782-a0fa-dec388c1b513 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:27:20.012559 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1014/2007] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:27:19 2022] POST /identity/v3/projects => generated 369 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:27:20.013737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff16f39-55e8-411d-a364-7f0e9d9f839e None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:27:20.024377 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff16f39-55e8-411d-a364-7f0e9d9f839e None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:27:20.038492 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:27:20.089147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c574a5-98cb-45f4-b063-7898e8cc6076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:27:20.089913 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c574a5-98cb-45f4-b063-7898e8cc6076 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:27:20.090286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c574a5-98cb-45f4-b063-7898e8cc6076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:27:20.090653 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c574a5-98cb-45f4-b063-7898e8cc6076 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:27:20.091384 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c574a5-98cb-45f4-b063-7898e8cc6076 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:27:20.093417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c574a5-98cb-45f4-b063-7898e8cc6076 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:27:20.118898 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 994/2008] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:27:20 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:27:20.120146 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169f9ee-5f1e-4782-a0fa-dec388c1b513 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:27:20.131203 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169f9ee-5f1e-4782-a0fa-dec388c1b513 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:27:20.137066 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78462ea-88ea-4020-a9bf-6d89d77248a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:27:20.137674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78462ea-88ea-4020-a9bf-6d89d77248a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:27:20.137950 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78462ea-88ea-4020-a9bf-6d89d77248a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:27:20.138220 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78462ea-88ea-4020-a9bf-6d89d77248a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:27:20.138856 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78462ea-88ea-4020-a9bf-6d89d77248a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:27:20.140438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78462ea-88ea-4020-a9bf-6d89d77248a7 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:27:20.153123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1015/2009] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:27:20.157479 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c574a5-98cb-45f4-b063-7898e8cc6076 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:27:20.168265 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c574a5-98cb-45f4-b063-7898e8cc6076 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:27:20.172977 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94dcc2f-be40-456c-b270-293a15d0fc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:27:20.173645 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94dcc2f-be40-456c-b270-293a15d0fc93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:27:20.173927 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94dcc2f-be40-456c-b270-293a15d0fc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:27:20.174204 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94dcc2f-be40-456c-b270-293a15d0fc93 admin admin] PATH_INFO: `/v3/projects/90e61099d1854a62b3d797a0c77e94f0/users/6a25a8cd8ff046f0ad04320a3c50d93a/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:27:20.193482 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94dcc2f-be40-456c-b270-293a15d0fc93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90e61099d1854a62b3d797a0c77e94f0, user_id=6a25a8cd8ff046f0ad04320a3c50d93a, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:27:20.197261 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94dcc2f-be40-456c-b270-293a15d0fc93 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:27:20.212801 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 995/2010] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:27:20 2022] PUT /identity/v3/projects/90e61099d1854a62b3d797a0c77e94f0/users/6a25a8cd8ff046f0ad04320a3c50d93a/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:27:20.216852 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78462ea-88ea-4020-a9bf-6d89d77248a7 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:27:20.227034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78462ea-88ea-4020-a9bf-6d89d77248a7 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:27:20.285103 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8df83b-a9b9-468f-bfb3-cf68645b8499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:27:20.285712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8df83b-a9b9-468f-bfb3-cf68645b8499 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:27:20.285987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8df83b-a9b9-468f-bfb3-cf68645b8499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:27:20.286255 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8df83b-a9b9-468f-bfb3-cf68645b8499 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:27:20.286885 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8df83b-a9b9-468f-bfb3-cf68645b8499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:27:20.288471 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8df83b-a9b9-468f-bfb3-cf68645b8499 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:27:20.298589 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1016/2011] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:27:20.303236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94dcc2f-be40-456c-b270-293a15d0fc93 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:27:20.316428 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94dcc2f-be40-456c-b270-293a15d0fc93 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:27:20.321939 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83991d5d-447b-4684-835b-a854ea86af73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:27:20.322658 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83991d5d-447b-4684-835b-a854ea86af73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:27:20.322933 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83991d5d-447b-4684-835b-a854ea86af73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:27:20.323204 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83991d5d-447b-4684-835b-a854ea86af73 admin admin] PATH_INFO: `/v3/projects/90e61099d1854a62b3d797a0c77e94f0/users/6a25a8cd8ff046f0ad04320a3c50d93a/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:27:20.325157 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83991d5d-447b-4684-835b-a854ea86af73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90e61099d1854a62b3d797a0c77e94f0, user_id=6a25a8cd8ff046f0ad04320a3c50d93a, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:27:20.327803 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83991d5d-447b-4684-835b-a854ea86af73 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:27:20.338861 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 996/2012] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:27:20 2022] PUT /identity/v3/projects/90e61099d1854a62b3d797a0c77e94f0/users/6a25a8cd8ff046f0ad04320a3c50d93a/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:27:20.350751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29910c8f-ed59-4a87-84f1-ec74b79e9408 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:27:20.351147 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29910c8f-ed59-4a87-84f1-ec74b79e9408 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:27:20.351488 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29910c8f-ed59-4a87-84f1-ec74b79e9408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:27:20.382290 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-29910c8f-ed59-4a87-84f1-ec74b79e9408 None None] MFA Rules not processed for user `6a25a8cd8ff046f0ad04320a3c50d93a`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:27:20.384225 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-29910c8f-ed59-4a87-84f1-ec74b79e9408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:27:20.453156 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1017/2013] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:27:20 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:27:30.334816 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=36,keystone:INSERT=5 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:27:30.450346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=92,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:28:07.751258 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83991d5d-447b-4684-835b-a854ea86af73 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:28:07.754571 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-83991d5d-447b-4684-835b-a854ea86af73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:28:07.831120 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83991d5d-447b-4684-835b-a854ea86af73 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:28:07.836660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d013fe-d305-451e-91cc-90789ed92a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:28:07.837520 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d013fe-d305-451e-91cc-90789ed92a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:28:07.837796 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d013fe-d305-451e-91cc-90789ed92a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:28:07.838065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d013fe-d305-451e-91cc-90789ed92a03 admin admin] PATH_INFO: `/v3/users/34992b551ea54a1fbe128e287e014847` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:28:07.839189 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d013fe-d305-451e-91cc-90789ed92a03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34992b551ea54a1fbe128e287e014847)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:28:07.841119 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d013fe-d305-451e-91cc-90789ed92a03 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:28:07.887853 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-51d013fe-d305-451e-91cc-90789ed92a03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34992b551ea54a1fbe128e287e014847'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:07.888505 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-51d013fe-d305-451e-91cc-90789ed92a03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34992b551ea54a1fbe128e287e014847'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:07.895960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-51d013fe-d305-451e-91cc-90789ed92a03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34992b551ea54a1fbe128e287e014847'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:07.908794 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-51d013fe-d305-451e-91cc-90789ed92a03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34992b551ea54a1fbe128e287e014847'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:07.916275 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-51d013fe-d305-451e-91cc-90789ed92a03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34992b551ea54a1fbe128e287e014847'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:07.922454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 997/2014] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:28:07 2022] DELETE /identity/v3/users/34992b551ea54a1fbe128e287e014847 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:28:08.096803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29910c8f-ed59-4a87-84f1-ec74b79e9408 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:28:08.110238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29910c8f-ed59-4a87-84f1-ec74b79e9408 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:28:08.174543 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f291b-56b7-49c5-9363-a6328f4d7a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:28:08.175122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f291b-56b7-49c5-9363-a6328f4d7a95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:28:08.175478 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f291b-56b7-49c5-9363-a6328f4d7a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:28:08.175776 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f291b-56b7-49c5-9363-a6328f4d7a95 admin admin] PATH_INFO: `/v3/projects/db144186ebfa49c185816e5c986b6098` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:28:08.177055 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f291b-56b7-49c5-9363-a6328f4d7a95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db144186ebfa49c185816e5c986b6098)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:28:08.179031 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f291b-56b7-49c5-9363-a6328f4d7a95 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:28:08.225984 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8c8f291b-56b7-49c5-9363-a6328f4d7a95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db144186ebfa49c185816e5c986b6098', 'request_id': 'req-8c8f291b-56b7-49c5-9363-a6328f4d7a95'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:08.231959 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8c8f291b-56b7-49c5-9363-a6328f4d7a95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db144186ebfa49c185816e5c986b6098', 'request_id': 'req-8c8f291b-56b7-49c5-9363-a6328f4d7a95'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:08.238791 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8c8f291b-56b7-49c5-9363-a6328f4d7a95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db144186ebfa49c185816e5c986b6098', 'request_id': 'req-8c8f291b-56b7-49c5-9363-a6328f4d7a95'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:08.251627 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8c8f291b-56b7-49c5-9363-a6328f4d7a95 admin admin] The token cache is being invalidate because project db144186ebfa49c185816e5c986b6098 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:28:08.251915 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8c8f291b-56b7-49c5-9363-a6328f4d7a95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:08.252539 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8c8f291b-56b7-49c5-9363-a6328f4d7a95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:08.254062 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1018/2015] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:28:08 2022] DELETE /identity/v3/projects/db144186ebfa49c185816e5c986b6098 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:28:08.258105 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d013fe-d305-451e-91cc-90789ed92a03 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:28:08.260338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-51d013fe-d305-451e-91cc-90789ed92a03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:28:08.273325 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d013fe-d305-451e-91cc-90789ed92a03 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:28:08.344250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5eabc7-fcf3-45a7-9505-faca92ff5f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:28:08.345219 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5eabc7-fcf3-45a7-9505-faca92ff5f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:28:08.345499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5eabc7-fcf3-45a7-9505-faca92ff5f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:28:08.345870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5eabc7-fcf3-45a7-9505-faca92ff5f0c admin admin] PATH_INFO: `/v3/users/03c353d72f4c4bde896ba11287614d1e` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:28:08.347197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5eabc7-fcf3-45a7-9505-faca92ff5f0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=03c353d72f4c4bde896ba11287614d1e)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:28:08.349235 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5eabc7-fcf3-45a7-9505-faca92ff5f0c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:28:08.371909 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:28:08.401028 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fe5eabc7-fcf3-45a7-9505-faca92ff5f0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03c353d72f4c4bde896ba11287614d1e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:08.401684 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fe5eabc7-fcf3-45a7-9505-faca92ff5f0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03c353d72f4c4bde896ba11287614d1e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:08.409178 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fe5eabc7-fcf3-45a7-9505-faca92ff5f0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03c353d72f4c4bde896ba11287614d1e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:08.423145 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fe5eabc7-fcf3-45a7-9505-faca92ff5f0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03c353d72f4c4bde896ba11287614d1e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:08.430499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fe5eabc7-fcf3-45a7-9505-faca92ff5f0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03c353d72f4c4bde896ba11287614d1e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:08.437608 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 998/2016] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:28:08 2022] DELETE /identity/v3/users/03c353d72f4c4bde896ba11287614d1e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:28:08.625065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f291b-56b7-49c5-9363-a6328f4d7a95 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:28:08.626645 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8c8f291b-56b7-49c5-9363-a6328f4d7a95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:28:08.641729 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f291b-56b7-49c5-9363-a6328f4d7a95 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:28:08.713160 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5deed-ef17-4d20-b092-880b9fec67ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:28:08.713901 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5deed-ef17-4d20-b092-880b9fec67ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:28:08.714251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5deed-ef17-4d20-b092-880b9fec67ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:28:08.714610 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5deed-ef17-4d20-b092-880b9fec67ea admin admin] PATH_INFO: `/v3/projects/08da307ec97b476f89e0be61ef9a6292` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:28:08.715835 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf5deed-ef17-4d20-b092-880b9fec67ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=08da307ec97b476f89e0be61ef9a6292)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:28:08.717844 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf5deed-ef17-4d20-b092-880b9fec67ea admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:28:08.754451 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:28:08.767796 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-3cf5deed-ef17-4d20-b092-880b9fec67ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08da307ec97b476f89e0be61ef9a6292', 'request_id': 'req-3cf5deed-ef17-4d20-b092-880b9fec67ea'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:08.773775 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-3cf5deed-ef17-4d20-b092-880b9fec67ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08da307ec97b476f89e0be61ef9a6292', 'request_id': 'req-3cf5deed-ef17-4d20-b092-880b9fec67ea'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:08.780401 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-3cf5deed-ef17-4d20-b092-880b9fec67ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08da307ec97b476f89e0be61ef9a6292', 'request_id': 'req-3cf5deed-ef17-4d20-b092-880b9fec67ea'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:08.794196 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-3cf5deed-ef17-4d20-b092-880b9fec67ea admin admin] The token cache is being invalidate because project 08da307ec97b476f89e0be61ef9a6292 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:28:08.794742 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-3cf5deed-ef17-4d20-b092-880b9fec67ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:08.795380 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-3cf5deed-ef17-4d20-b092-880b9fec67ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:08.796912 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1019/2017] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:28:08 2022] DELETE /identity/v3/projects/08da307ec97b476f89e0be61ef9a6292 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:28:09.447275 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5eabc7-fcf3-45a7-9505-faca92ff5f0c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:28:09.448351 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fe5eabc7-fcf3-45a7-9505-faca92ff5f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:28:09.542420 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5eabc7-fcf3-45a7-9505-faca92ff5f0c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:28:09.594128 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2283e-b2f8-4a8c-8b4c-29381addca85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:28:09.594744 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2283e-b2f8-4a8c-8b4c-29381addca85 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:28:09.595614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2283e-b2f8-4a8c-8b4c-29381addca85 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:28:09.595885 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2283e-b2f8-4a8c-8b4c-29381addca85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:28:09.597620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-21f2283e-b2f8-4a8c-8b4c-29381addca85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:28:09.668004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:28:09.675978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2283e-b2f8-4a8c-8b4c-29381addca85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:28:09.677821 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2283e-b2f8-4a8c-8b4c-29381addca85 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:28:09.689556 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 999/2018] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:28:09 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:28:10.020233 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5deed-ef17-4d20-b092-880b9fec67ea admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:28:10.022859 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3cf5deed-ef17-4d20-b092-880b9fec67ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:28:10.138911 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5deed-ef17-4d20-b092-880b9fec67ea admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:28:10.144717 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e7a89-4210-40fb-a374-e667a71336e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:28:10.145619 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e7a89-4210-40fb-a374-e667a71336e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:28:10.146016 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e7a89-4210-40fb-a374-e667a71336e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:28:10.146392 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e7a89-4210-40fb-a374-e667a71336e7 admin admin] PATH_INFO: `/v3/users/1baa5e519aac48a49dac5a452ae07694` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:28:10.147712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e7a89-4210-40fb-a374-e667a71336e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1baa5e519aac48a49dac5a452ae07694)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:28:10.150598 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e7a89-4210-40fb-a374-e667a71336e7 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:28:10.215809 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a90e7a89-4210-40fb-a374-e667a71336e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1baa5e519aac48a49dac5a452ae07694'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:10.217178 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a90e7a89-4210-40fb-a374-e667a71336e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1baa5e519aac48a49dac5a452ae07694'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:10.229349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a90e7a89-4210-40fb-a374-e667a71336e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1baa5e519aac48a49dac5a452ae07694'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:10.243940 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a90e7a89-4210-40fb-a374-e667a71336e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1baa5e519aac48a49dac5a452ae07694'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:10.244753 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a90e7a89-4210-40fb-a374-e667a71336e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1baa5e519aac48a49dac5a452ae07694'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:10.265768 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1020/2019] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:28:10 2022] DELETE /identity/v3/users/1baa5e519aac48a49dac5a452ae07694 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:28:10.270483 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e634c-d3c4-4ef0-ae33-ca839b27c5d7 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:28:10.270870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e634c-d3c4-4ef0-ae33-ca839b27c5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:28:10.271294 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e634c-d3c4-4ef0-ae33-ca839b27c5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:28:10.294955 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-714e634c-d3c4-4ef0-ae33-ca839b27c5d7 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:28:10.300043 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-714e634c-d3c4-4ef0-ae33-ca839b27c5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:28:10.385104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1000/2020] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:28:10.396212 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e7a89-4210-40fb-a374-e667a71336e7 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:28:10.398996 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a90e7a89-4210-40fb-a374-e667a71336e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:28:10.453140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:28:10.474121 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e7a89-4210-40fb-a374-e667a71336e7 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:28:10.478074 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc3f17-e3ed-4909-b2ac-26652ab8cd83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:28:10.478840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc3f17-e3ed-4909-b2ac-26652ab8cd83 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:28:10.479187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc3f17-e3ed-4909-b2ac-26652ab8cd83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:28:10.479525 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc3f17-e3ed-4909-b2ac-26652ab8cd83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:28:10.489285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cc3f17-e3ed-4909-b2ac-26652ab8cd83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:28:10.490813 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cc3f17-e3ed-4909-b2ac-26652ab8cd83 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:28:10.503398 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1021/2021] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:28:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:28:10.823019 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e634c-d3c4-4ef0-ae33-ca839b27c5d7 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:28:10.823019 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-714e634c-d3c4-4ef0-ae33-ca839b27c5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:28:10.836532 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e634c-d3c4-4ef0-ae33-ca839b27c5d7 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:28:10.841962 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18dbf8-4e6c-4adc-b716-87ffda5c5075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:28:10.842614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18dbf8-4e6c-4adc-b716-87ffda5c5075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:28:10.842894 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18dbf8-4e6c-4adc-b716-87ffda5c5075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:28:10.843168 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18dbf8-4e6c-4adc-b716-87ffda5c5075 admin admin] PATH_INFO: `/v3/projects/31067798d7384f4489c25dd5660b823f` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:28:10.844300 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae18dbf8-4e6c-4adc-b716-87ffda5c5075 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31067798d7384f4489c25dd5660b823f)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:28:10.846462 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae18dbf8-4e6c-4adc-b716-87ffda5c5075 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:28:10.909595 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ae18dbf8-4e6c-4adc-b716-87ffda5c5075 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31067798d7384f4489c25dd5660b823f', 'request_id': 'req-ae18dbf8-4e6c-4adc-b716-87ffda5c5075'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:10.916217 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ae18dbf8-4e6c-4adc-b716-87ffda5c5075 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31067798d7384f4489c25dd5660b823f', 'request_id': 'req-ae18dbf8-4e6c-4adc-b716-87ffda5c5075'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:10.922711 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ae18dbf8-4e6c-4adc-b716-87ffda5c5075 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31067798d7384f4489c25dd5660b823f', 'request_id': 'req-ae18dbf8-4e6c-4adc-b716-87ffda5c5075'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:10.940463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ae18dbf8-4e6c-4adc-b716-87ffda5c5075 admin admin] The token cache is being invalidate because project 31067798d7384f4489c25dd5660b823f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:28:10.940789 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ae18dbf8-4e6c-4adc-b716-87ffda5c5075 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:10.941467 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ae18dbf8-4e6c-4adc-b716-87ffda5c5075 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:28:10.942963 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1001/2022] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:28:10 2022] DELETE /identity/v3/projects/31067798d7384f4489c25dd5660b823f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:28:10.949920 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c35d90-1f11-4c17-9121-111fb5859beb None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:28:10.950524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c35d90-1f11-4c17-9121-111fb5859beb None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:28:10.950890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c35d90-1f11-4c17-9121-111fb5859beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:28:10.971491 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-86c35d90-1f11-4c17-9121-111fb5859beb None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:28:10.973077 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-86c35d90-1f11-4c17-9121-111fb5859beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:28:11.044259 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1022/2023] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:28:11.053650 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a68522-e1bd-4d67-8139-57961eafa1be None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:28:11.053935 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a68522-e1bd-4d67-8139-57961eafa1be None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:28:11.054325 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a68522-e1bd-4d67-8139-57961eafa1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:28:11.077681 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c3a68522-e1bd-4d67-8139-57961eafa1be None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:28:11.078880 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c3a68522-e1bd-4d67-8139-57961eafa1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:28:11.087054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1002/2024] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:28:11.091565 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c35d90-1f11-4c17-9121-111fb5859beb None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:28:11.106224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c35d90-1f11-4c17-9121-111fb5859beb None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:28:11.111271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410756cb-4439-4b8f-a2f3-843b9b0509b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:28:11.111902 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410756cb-4439-4b8f-a2f3-843b9b0509b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:28:11.112184 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410756cb-4439-4b8f-a2f3-843b9b0509b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:28:11.112453 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410756cb-4439-4b8f-a2f3-843b9b0509b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:28:11.113143 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410756cb-4439-4b8f-a2f3-843b9b0509b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:28:11.113447 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410756cb-4439-4b8f-a2f3-843b9b0509b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:28:11.115081 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410756cb-4439-4b8f-a2f3-843b9b0509b5 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:28:11.128175 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1023/2025] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:28:11.133249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a68522-e1bd-4d67-8139-57961eafa1be None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:28:11.144886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a68522-e1bd-4d67-8139-57961eafa1be None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:28:11.149155 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324de7b-a65f-439b-b0b6-020d8e4d4ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:28:11.150364 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324de7b-a65f-439b-b0b6-020d8e4d4ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:28:11.150712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324de7b-a65f-439b-b0b6-020d8e4d4ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:28:11.151086 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324de7b-a65f-439b-b0b6-020d8e4d4ff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:28:11.151801 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a324de7b-a65f-439b-b0b6-020d8e4d4ff8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:28:11.153808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a324de7b-a65f-439b-b0b6-020d8e4d4ff8 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:28:11.182187 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1003/2026] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:28:11 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:28:11.186480 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410756cb-4439-4b8f-a2f3-843b9b0509b5 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:28:11.196394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410756cb-4439-4b8f-a2f3-843b9b0509b5 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:28:11.258390 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cd1e0-8ca9-4fe9-8d46-adf979d0b16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:28:11.260244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cd1e0-8ca9-4fe9-8d46-adf979d0b16f admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:28:11.260522 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cd1e0-8ca9-4fe9-8d46-adf979d0b16f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:28:11.260819 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cd1e0-8ca9-4fe9-8d46-adf979d0b16f admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:28:11.261537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393cd1e0-8ca9-4fe9-8d46-adf979d0b16f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:28:11.263471 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393cd1e0-8ca9-4fe9-8d46-adf979d0b16f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:28:11.290846 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1024/2027] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:28:11 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:28:11.292393 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324de7b-a65f-439b-b0b6-020d8e4d4ff8 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:28:11.303905 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324de7b-a65f-439b-b0b6-020d8e4d4ff8 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:28:11.309429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a40b1-2ace-48cb-bf17-fe164f50297e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:28:11.310042 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a40b1-2ace-48cb-bf17-fe164f50297e admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:28:11.310319 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a40b1-2ace-48cb-bf17-fe164f50297e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:28:11.310588 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a40b1-2ace-48cb-bf17-fe164f50297e admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:28:11.311224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a40b1-2ace-48cb-bf17-fe164f50297e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:28:11.312845 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a40b1-2ace-48cb-bf17-fe164f50297e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:28:11.323636 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1004/2028] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:28:11.327875 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cd1e0-8ca9-4fe9-8d46-adf979d0b16f admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:28:11.338732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cd1e0-8ca9-4fe9-8d46-adf979d0b16f admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:28:11.344248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818168a2-6003-4515-9669-d83433e1e4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:28:11.344934 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818168a2-6003-4515-9669-d83433e1e4ed admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:28:11.345214 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818168a2-6003-4515-9669-d83433e1e4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:28:11.345486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818168a2-6003-4515-9669-d83433e1e4ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:28:11.346145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818168a2-6003-4515-9669-d83433e1e4ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:28:11.347776 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818168a2-6003-4515-9669-d83433e1e4ed admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:28:11.360022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1025/2029] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:28:11.364294 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a40b1-2ace-48cb-bf17-fe164f50297e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:28:11.375971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a40b1-2ace-48cb-bf17-fe164f50297e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:28:11.381051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986b5db-0f0d-4780-a8b1-3f3dfec7311d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:28:11.381741 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986b5db-0f0d-4780-a8b1-3f3dfec7311d admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:28:11.382024 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986b5db-0f0d-4780-a8b1-3f3dfec7311d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:28:11.382302 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986b5db-0f0d-4780-a8b1-3f3dfec7311d admin admin] PATH_INFO: `/v3/projects/87f5b76358024015a9bf1d9a49a4f222/users/6efdaa628f0644228d51686cc1a6bbdc/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:28:11.400811 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986b5db-0f0d-4780-a8b1-3f3dfec7311d admin admin] RBAC: Authorizing `identity:create_grant(project_id=87f5b76358024015a9bf1d9a49a4f222, user_id=6efdaa628f0644228d51686cc1a6bbdc, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:28:11.403533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986b5db-0f0d-4780-a8b1-3f3dfec7311d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:28:11.419277 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1005/2030] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:28:11 2022] PUT /identity/v3/projects/87f5b76358024015a9bf1d9a49a4f222/users/6efdaa628f0644228d51686cc1a6bbdc/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:28:11.428526 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58bca3-9362-4b5f-b1b2-9e85209708dc None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:28:11.428862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58bca3-9362-4b5f-b1b2-9e85209708dc None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:28:11.429131 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58bca3-9362-4b5f-b1b2-9e85209708dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:28:11.457940 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-dd58bca3-9362-4b5f-b1b2-9e85209708dc None None] MFA Rules not processed for user `6efdaa628f0644228d51686cc1a6bbdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:28:11.459154 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-dd58bca3-9362-4b5f-b1b2-9e85209708dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:28:11.499160 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:28:11.529578 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1026/2031] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:28:11 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:28:21.413132 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:28:21.527203 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=10 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:28:36.664392 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986b5db-0f0d-4780-a8b1-3f3dfec7311d admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:28:36.666179 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-9986b5db-0f0d-4780-a8b1-3f3dfec7311d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:28:36.725009 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986b5db-0f0d-4780-a8b1-3f3dfec7311d admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:28:36.728510 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f620d-1417-41b6-837f-47e534116362 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:28:36.729271 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f620d-1417-41b6-837f-47e534116362 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:28:36.729424 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f620d-1417-41b6-837f-47e534116362 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:28:36.729668 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f620d-1417-41b6-837f-47e534116362 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:28:36.731386 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-576f620d-1417-41b6-837f-47e534116362 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:28:36.792726 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f620d-1417-41b6-837f-47e534116362 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:28:36.794222 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f620d-1417-41b6-837f-47e534116362 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:28:36.804727 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1006/2032] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:28:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:28:37.169437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58bca3-9362-4b5f-b1b2-9e85209708dc None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:28:37.182474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58bca3-9362-4b5f-b1b2-9e85209708dc None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:28:37.186061 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc186456-c84f-46fc-8a62-dba586b11988 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:28:37.186640 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc186456-c84f-46fc-8a62-dba586b11988 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:28:37.186962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc186456-c84f-46fc-8a62-dba586b11988 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:28:37.187361 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc186456-c84f-46fc-8a62-dba586b11988 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:28:37.196279 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc186456-c84f-46fc-8a62-dba586b11988 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:28:37.198701 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc186456-c84f-46fc-8a62-dba586b11988 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:28:37.214387 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1027/2033] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:28:37 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:28:43.432462 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f620d-1417-41b6-837f-47e534116362 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:28:43.434472 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-576f620d-1417-41b6-837f-47e534116362 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:28:43.490636 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f620d-1417-41b6-837f-47e534116362 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:28:43.494359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec00584-4f7f-4358-a7b0-b0e2b03cf35d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:28:43.494968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec00584-4f7f-4358-a7b0-b0e2b03cf35d service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:28:43.495878 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec00584-4f7f-4358-a7b0-b0e2b03cf35d service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:28:43.496164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec00584-4f7f-4358-a7b0-b0e2b03cf35d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:28:43.498089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0ec00584-4f7f-4358-a7b0-b0e2b03cf35d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:28:43.567834 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:28:43.570423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec00584-4f7f-4358-a7b0-b0e2b03cf35d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:28:43.572505 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec00584-4f7f-4358-a7b0-b0e2b03cf35d service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:28:43.584465 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1007/2034] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:28:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:28:47.203999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=6 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:28:53.579749 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:28:54.426404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc186456-c84f-46fc-8a62-dba586b11988 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:28:54.428561 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-fc186456-c84f-46fc-8a62-dba586b11988 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:28:54.504593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc186456-c84f-46fc-8a62-dba586b11988 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:28:54.510082 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f78d69-e5a2-4d88-9bcb-7399fb5ac577 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfbaaf32475a4c82a34948602d8f4e35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:28:54.511123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f78d69-e5a2-4d88-9bcb-7399fb5ac577 service placement] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:28:54.511406 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f78d69-e5a2-4d88-9bcb-7399fb5ac577 service placement] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:28:54.511680 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f78d69-e5a2-4d88-9bcb-7399fb5ac577 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:28:54.513540 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-39f78d69-e5a2-4d88-9bcb-7399fb5ac577 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:28:54.525761 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f78d69-e5a2-4d88-9bcb-7399fb5ac577 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:28:54.527595 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f78d69-e5a2-4d88-9bcb-7399fb5ac577 service placement] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:28:54.544939 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1028/2035] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:28:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:29:04.533746 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=32 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:30:10.382110 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec00584-4f7f-4358-a7b0-b0e2b03cf35d service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:30:10.383518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0ec00584-4f7f-4358-a7b0-b0e2b03cf35d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:30:10.395887 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec00584-4f7f-4358-a7b0-b0e2b03cf35d service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:30:10.400227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7823d24f-6d44-413a-8b6b-b31055bb40a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:30:10.401084 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823d24f-6d44-413a-8b6b-b31055bb40a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:30:10.401365 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823d24f-6d44-413a-8b6b-b31055bb40a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:30:10.401796 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823d24f-6d44-413a-8b6b-b31055bb40a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:30:10.404036 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7823d24f-6d44-413a-8b6b-b31055bb40a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:30:10.471235 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7823d24f-6d44-413a-8b6b-b31055bb40a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:30:10.472674 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7823d24f-6d44-413a-8b6b-b31055bb40a5 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:30:10.482308 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1008/2036] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:30:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:30:20.477625 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=28 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:30:35.726908 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f78d69-e5a2-4d88-9bcb-7399fb5ac577 service placement] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:30:35.728454 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-39f78d69-e5a2-4d88-9bcb-7399fb5ac577 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:30:35.740614 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f78d69-e5a2-4d88-9bcb-7399fb5ac577 service placement] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:30:35.744190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a229d1-8e2d-4b8a-80af-f55930b6c01a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:30:35.744888 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a229d1-8e2d-4b8a-80af-f55930b6c01a service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:30:35.745696 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a229d1-8e2d-4b8a-80af-f55930b6c01a service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:30:35.745696 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a229d1-8e2d-4b8a-80af-f55930b6c01a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:30:35.747471 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-97a229d1-8e2d-4b8a-80af-f55930b6c01a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:30:35.763013 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a229d1-8e2d-4b8a-80af-f55930b6c01a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:30:35.764648 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a229d1-8e2d-4b8a-80af-f55930b6c01a service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:30:35.777102 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1029/2037] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:30:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:30:45.771126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=6 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:30:57.101187 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7823d24f-6d44-413a-8b6b-b31055bb40a5 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:30:57.111222 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7823d24f-6d44-413a-8b6b-b31055bb40a5 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:30:57.114844 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413e538-cb0b-46a2-b2f8-79a57944f254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:30:57.115449 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413e538-cb0b-46a2-b2f8-79a57944f254 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:30:57.115729 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413e538-cb0b-46a2-b2f8-79a57944f254 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:30:57.116001 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413e538-cb0b-46a2-b2f8-79a57944f254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:30:57.117872 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f413e538-cb0b-46a2-b2f8-79a57944f254 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:30:57.128539 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f413e538-cb0b-46a2-b2f8-79a57944f254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:30:57.130068 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f413e538-cb0b-46a2-b2f8-79a57944f254 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:30:57.140399 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1009/2038] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:30:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:31:07.134664 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=6 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:31:09.020146 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a229d1-8e2d-4b8a-80af-f55930b6c01a service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:31:09.021774 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-97a229d1-8e2d-4b8a-80af-f55930b6c01a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:31:09.035943 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a229d1-8e2d-4b8a-80af-f55930b6c01a service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:31:09.041800 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e7597-aee2-4227-b1de-e4eb1d1c49ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:31:09.042906 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e7597-aee2-4227-b1de-e4eb1d1c49ad service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:31:09.043185 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e7597-aee2-4227-b1de-e4eb1d1c49ad service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:31:09.043491 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e7597-aee2-4227-b1de-e4eb1d1c49ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:31:09.045281 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-670e7597-aee2-4227-b1de-e4eb1d1c49ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:31:09.118241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e7597-aee2-4227-b1de-e4eb1d1c49ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:31:09.119824 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e7597-aee2-4227-b1de-e4eb1d1c49ad service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:31:09.129404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1030/2039] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:31:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:31:19.124989 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=33 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:32:11.280901 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413e538-cb0b-46a2-b2f8-79a57944f254 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:32:11.291210 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413e538-cb0b-46a2-b2f8-79a57944f254 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:32:11.294766 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d2633-2e20-4d56-9e6f-dd9d7ae61ac6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:32:11.295375 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d2633-2e20-4d56-9e6f-dd9d7ae61ac6 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:32:11.296210 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d2633-2e20-4d56-9e6f-dd9d7ae61ac6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:32:11.296484 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d2633-2e20-4d56-9e6f-dd9d7ae61ac6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:32:11.298426 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-cb9d2633-2e20-4d56-9e6f-dd9d7ae61ac6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:32:11.382609 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d2633-2e20-4d56-9e6f-dd9d7ae61ac6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:32:11.384481 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d2633-2e20-4d56-9e6f-dd9d7ae61ac6 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:32:11.403582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1010/2040] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:32:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:32:11.430574 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e7597-aee2-4227-b1de-e4eb1d1c49ad service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:32:11.440112 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e7597-aee2-4227-b1de-e4eb1d1c49ad service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:32:11.443689 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3e56b-2802-43d4-b45e-0378a93029e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:32:11.444293 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3e56b-2802-43d4-b45e-0378a93029e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:32:11.444573 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3e56b-2802-43d4-b45e-0378a93029e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:32:11.444904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3e56b-2802-43d4-b45e-0378a93029e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:32:11.446662 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-59d3e56b-2802-43d4-b45e-0378a93029e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:32:11.518818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d3e56b-2802-43d4-b45e-0378a93029e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:32:11.520395 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d3e56b-2802-43d4-b45e-0378a93029e1 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:32:11.530492 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1031/2041] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:32:11 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:32:21.400830 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=41 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:32:21.525854 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=33 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:33:45.556358 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d2633-2e20-4d56-9e6f-dd9d7ae61ac6 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:33:45.565760 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d2633-2e20-4d56-9e6f-dd9d7ae61ac6 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:33:45.569724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b6e3f-d488-491c-9aad-3374b423e82a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:33:45.570337 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b6e3f-d488-491c-9aad-3374b423e82a service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:33:45.570620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b6e3f-d488-491c-9aad-3374b423e82a service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:33:45.570895 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b6e3f-d488-491c-9aad-3374b423e82a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:33:45.603349 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b6e3f-d488-491c-9aad-3374b423e82a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:33:45.604999 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b6e3f-d488-491c-9aad-3374b423e82a service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:33:45.616912 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1011/2042] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:33:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:33:55.611158 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=15 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:34:00.476803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3e56b-2802-43d4-b45e-0378a93029e1 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:34:00.478219 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-59d3e56b-2802-43d4-b45e-0378a93029e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:34:00.557361 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3e56b-2802-43d4-b45e-0378a93029e1 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:34:00.562374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf9f3d-0d78-420e-953f-d0a5c8fd90f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfbaaf32475a4c82a34948602d8f4e35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:34:00.562976 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf9f3d-0d78-420e-953f-d0a5c8fd90f9 service placement] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:34:00.563253 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf9f3d-0d78-420e-953f-d0a5c8fd90f9 service placement] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:34:00.563522 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf9f3d-0d78-420e-953f-d0a5c8fd90f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:34:00.572303 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf9f3d-0d78-420e-953f-d0a5c8fd90f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:34:00.574496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf9f3d-0d78-420e-953f-d0a5c8fd90f9 service placement] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:34:00.584446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1032/2043] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:34:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:34:10.580118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=34 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:34:11.784088 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b6e3f-d488-491c-9aad-3374b423e82a service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:34:11.812330 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b6e3f-d488-491c-9aad-3374b423e82a service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:34:11.816143 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65468a70-ea1f-4ea3-84f5-b3ca1de5bfaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:34:11.816777 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65468a70-ea1f-4ea3-84f5-b3ca1de5bfaf service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:34:11.817056 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65468a70-ea1f-4ea3-84f5-b3ca1de5bfaf service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:34:11.817352 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65468a70-ea1f-4ea3-84f5-b3ca1de5bfaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:34:11.827379 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65468a70-ea1f-4ea3-84f5-b3ca1de5bfaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:34:11.828869 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65468a70-ea1f-4ea3-84f5-b3ca1de5bfaf service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:34:11.839215 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1012/2044] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:34:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:34:12.253836 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf9f3d-0d78-420e-953f-d0a5c8fd90f9 service placement] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:34:12.265712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf9f3d-0d78-420e-953f-d0a5c8fd90f9 service placement] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:34:12.270264 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411a703-a30d-4cf1-9c9a-b61109a7ca97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:34:12.272683 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411a703-a30d-4cf1-9c9a-b61109a7ca97 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:34:12.273199 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411a703-a30d-4cf1-9c9a-b61109a7ca97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:34:12.273637 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411a703-a30d-4cf1-9c9a-b61109a7ca97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:34:12.286068 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411a703-a30d-4cf1-9c9a-b61109a7ca97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:34:12.288193 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411a703-a30d-4cf1-9c9a-b61109a7ca97 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:34:12.311958 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1033/2045] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:34:12 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:34:12.423562 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8b9d2-1688-4f38-97e1-6ba814182933 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:34:12.423840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8b9d2-1688-4f38-97e1-6ba814182933 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:34:12.424318 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8b9d2-1688-4f38-97e1-6ba814182933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:34:12.455307 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-a4c8b9d2-1688-4f38-97e1-6ba814182933 None None] MFA Rules not processed for user `6da8a7cb6ef849faad7d7091a1095ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:34:12.456484 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a4c8b9d2-1688-4f38-97e1-6ba814182933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:34:12.463475 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1013/2046] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:34:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:34:12.848443 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411a703-a30d-4cf1-9c9a-b61109a7ca97 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:34:12.858420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411a703-a30d-4cf1-9c9a-b61109a7ca97 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:34:12.862139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af6eff-9e8f-4ef3-9668-3d33e78b20a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:34:12.862733 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af6eff-9e8f-4ef3-9668-3d33e78b20a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:34:12.863005 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af6eff-9e8f-4ef3-9668-3d33e78b20a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:34:12.863323 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af6eff-9e8f-4ef3-9668-3d33e78b20a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:34:12.865959 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-38af6eff-9e8f-4ef3-9668-3d33e78b20a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:34:12.877428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af6eff-9e8f-4ef3-9668-3d33e78b20a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:34:12.879097 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af6eff-9e8f-4ef3-9668-3d33e78b20a1 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:34:12.899520 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1034/2047] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:34:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:34:13.230655 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8b9d2-1688-4f38-97e1-6ba814182933 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:34:13.243490 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8b9d2-1688-4f38-97e1-6ba814182933 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:34:13.248585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395628df-45fa-446b-9aff-bc9e322e8841 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfbaaf32475a4c82a34948602d8f4e35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:34:13.249287 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395628df-45fa-446b-9aff-bc9e322e8841 service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:34:13.249633 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395628df-45fa-446b-9aff-bc9e322e8841 service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:34:13.250149 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395628df-45fa-446b-9aff-bc9e322e8841 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:34:13.252177 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-395628df-45fa-446b-9aff-bc9e322e8841 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:34:13.267251 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395628df-45fa-446b-9aff-bc9e322e8841 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:34:13.269258 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395628df-45fa-446b-9aff-bc9e322e8841 service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:34:13.281635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1014/2048] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:34:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:34:15.875065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af6eff-9e8f-4ef3-9668-3d33e78b20a1 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:34:15.893168 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af6eff-9e8f-4ef3-9668-3d33e78b20a1 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:34:15.899985 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f54e71-e3d3-487a-ae37-242bc1ad7794 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:34:15.900676 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f54e71-e3d3-487a-ae37-242bc1ad7794 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:34:15.901735 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f54e71-e3d3-487a-ae37-242bc1ad7794 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:34:15.902212 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f54e71-e3d3-487a-ae37-242bc1ad7794 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:34:15.904208 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f3f54e71-e3d3-487a-ae37-242bc1ad7794 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:34:15.920163 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f54e71-e3d3-487a-ae37-242bc1ad7794 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:34:15.922664 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f54e71-e3d3-487a-ae37-242bc1ad7794 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:34:15.936942 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1035/2049] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:34:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:34:21.217781 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395628df-45fa-446b-9aff-bc9e322e8841 service placement] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:34:21.234348 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395628df-45fa-446b-9aff-bc9e322e8841 service placement] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:34:21.241502 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5403af-6f5f-40bc-bff7-73ce350fb9f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:34:21.242554 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5403af-6f5f-40bc-bff7-73ce350fb9f5 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:34:21.243011 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5403af-6f5f-40bc-bff7-73ce350fb9f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:34:21.243450 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5403af-6f5f-40bc-bff7-73ce350fb9f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:34:21.245580 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7f5403af-6f5f-40bc-bff7-73ce350fb9f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:34:21.263935 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5403af-6f5f-40bc-bff7-73ce350fb9f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:34:21.265642 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5403af-6f5f-40bc-bff7-73ce350fb9f5 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:34:21.288905 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1015/2050] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:34:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:34:25.930303 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=18 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:34:31.272849 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=29 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:34:42.626332 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabcf53-893b-4715-a692-1149a430b3ab None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:34:42.626925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabcf53-893b-4715-a692-1149a430b3ab None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:34:42.626925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabcf53-893b-4715-a692-1149a430b3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:34:42.648561 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-afabcf53-893b-4715-a692-1149a430b3ab None None] MFA Rules not processed for user `2c6be23065c643939055a3de375187d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:34:42.649870 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-afabcf53-893b-4715-a692-1149a430b3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:34:42.657277 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1036/2051] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:34:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:34:52.643116 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:34:55.092448 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5403af-6f5f-40bc-bff7-73ce350fb9f5 service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:34:55.124564 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5403af-6f5f-40bc-bff7-73ce350fb9f5 service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:34:55.130177 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b720bb3-af11-4b57-bc4a-358abfc8dc72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:34:55.130891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b720bb3-af11-4b57-bc4a-358abfc8dc72 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:34:55.131393 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b720bb3-af11-4b57-bc4a-358abfc8dc72 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:34:55.131746 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b720bb3-af11-4b57-bc4a-358abfc8dc72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:34:55.133876 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3b720bb3-af11-4b57-bc4a-358abfc8dc72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:34:55.146502 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b720bb3-af11-4b57-bc4a-358abfc8dc72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:34:55.148284 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b720bb3-af11-4b57-bc4a-358abfc8dc72 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:34:55.159551 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1016/2052] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:34:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:35:05.161263 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=13 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:35:31.032806 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afabcf53-893b-4715-a692-1149a430b3ab None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:35:31.047671 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afabcf53-893b-4715-a692-1149a430b3ab None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:35:31.053002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345599c3-4839-445b-b6a3-b5c5de5df6a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:35:31.053842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345599c3-4839-445b-b6a3-b5c5de5df6a0 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:35:31.054112 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345599c3-4839-445b-b6a3-b5c5de5df6a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:35:31.054372 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345599c3-4839-445b-b6a3-b5c5de5df6a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:35:31.062739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345599c3-4839-445b-b6a3-b5c5de5df6a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:35:31.064298 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345599c3-4839-445b-b6a3-b5c5de5df6a0 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:35:31.073824 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1037/2053] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:35:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:35:41.068943 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=6 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:36:00.094716 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b720bb3-af11-4b57-bc4a-358abfc8dc72 service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:36:00.113077 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b720bb3-af11-4b57-bc4a-358abfc8dc72 service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:36:00.116644 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c440c1c-3c6e-4f94-ae0a-74539ce8e438 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:36:00.117265 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c440c1c-3c6e-4f94-ae0a-74539ce8e438 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:36:00.117543 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c440c1c-3c6e-4f94-ae0a-74539ce8e438 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:36:00.117812 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c440c1c-3c6e-4f94-ae0a-74539ce8e438 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:36:00.124904 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c440c1c-3c6e-4f94-ae0a-74539ce8e438 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:36:00.126326 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c440c1c-3c6e-4f94-ae0a-74539ce8e438 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:36:00.134949 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1017/2054] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:36:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:36:03.389689 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345599c3-4839-445b-b6a3-b5c5de5df6a0 service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:36:03.399087 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345599c3-4839-445b-b6a3-b5c5de5df6a0 service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:36:03.402730 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdfb65-7173-4853-b39b-527a27714a28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:36:03.403317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdfb65-7173-4853-b39b-527a27714a28 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:36:03.403587 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdfb65-7173-4853-b39b-527a27714a28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:36:03.403857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdfb65-7173-4853-b39b-527a27714a28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:36:03.413899 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cdfb65-7173-4853-b39b-527a27714a28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:36:03.415440 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cdfb65-7173-4853-b39b-527a27714a28 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:36:03.426348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1038/2055] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:36:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:36:10.130641 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=10 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:36:13.049098 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c440c1c-3c6e-4f94-ae0a-74539ce8e438 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:36:13.068498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c440c1c-3c6e-4f94-ae0a-74539ce8e438 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:36:13.072083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e874b-7dbd-4303-b6a3-b31f06eca6d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:36:13.072673 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e874b-7dbd-4303-b6a3-b31f06eca6d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:36:13.072976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e874b-7dbd-4303-b6a3-b31f06eca6d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:36:13.073247 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e874b-7dbd-4303-b6a3-b31f06eca6d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:36:13.081411 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e874b-7dbd-4303-b6a3-b31f06eca6d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:36:13.082819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e874b-7dbd-4303-b6a3-b31f06eca6d4 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:36:13.109766 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1018/2056] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:36:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:36:13.421544 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=6 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:36:13.764253 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdfb65-7173-4853-b39b-527a27714a28 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:36:13.772662 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdfb65-7173-4853-b39b-527a27714a28 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:36:13.776299 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9663d-79c7-4e5f-bf67-ee5cb8d63a4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:36:13.776920 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9663d-79c7-4e5f-bf67-ee5cb8d63a4d service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:36:13.777200 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9663d-79c7-4e5f-bf67-ee5cb8d63a4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:36:13.777474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9663d-79c7-4e5f-bf67-ee5cb8d63a4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:36:13.779166 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a1b9663d-79c7-4e5f-bf67-ee5cb8d63a4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:36:13.864650 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b9663d-79c7-4e5f-bf67-ee5cb8d63a4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:36:13.866126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b9663d-79c7-4e5f-bf67-ee5cb8d63a4d service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:36:13.886658 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1039/2057] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:36:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:36:16.945997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e874b-7dbd-4303-b6a3-b31f06eca6d4 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:36:16.959088 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e874b-7dbd-4303-b6a3-b31f06eca6d4 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:36:16.967419 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76746ee-0ff3-4ad1-ad8c-facf4a64a1c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfbaaf32475a4c82a34948602d8f4e35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:36:16.968211 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76746ee-0ff3-4ad1-ad8c-facf4a64a1c6 service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:36:16.968595 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76746ee-0ff3-4ad1-ad8c-facf4a64a1c6 service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:36:16.969003 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76746ee-0ff3-4ad1-ad8c-facf4a64a1c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:36:16.970903 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a76746ee-0ff3-4ad1-ad8c-facf4a64a1c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:36:16.987325 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76746ee-0ff3-4ad1-ad8c-facf4a64a1c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:36:16.989319 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76746ee-0ff3-4ad1-ad8c-facf4a64a1c6 service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:36:16.997328 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9663d-79c7-4e5f-bf67-ee5cb8d63a4d service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:36:17.005254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a1b9663d-79c7-4e5f-bf67-ee5cb8d63a4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:36:17.005661 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1019/2058] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:36:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:36:17.024132 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9663d-79c7-4e5f-bf67-ee5cb8d63a4d service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:36:17.030797 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e890a6-6806-4d97-97be-d5f9a10385d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:36:17.031484 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e890a6-6806-4d97-97be-d5f9a10385d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:36:17.031788 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e890a6-6806-4d97-97be-d5f9a10385d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:36:17.032060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e890a6-6806-4d97-97be-d5f9a10385d7 admin admin] PATH_INFO: `/v3/users/db34ffc8f90247daac1548bbb421b015` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:36:17.033319 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e890a6-6806-4d97-97be-d5f9a10385d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db34ffc8f90247daac1548bbb421b015)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:36:17.035243 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e890a6-6806-4d97-97be-d5f9a10385d7 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:36:17.087353 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-60e890a6-6806-4d97-97be-d5f9a10385d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db34ffc8f90247daac1548bbb421b015'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:36:17.088000 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-60e890a6-6806-4d97-97be-d5f9a10385d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db34ffc8f90247daac1548bbb421b015'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:36:17.095764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-60e890a6-6806-4d97-97be-d5f9a10385d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db34ffc8f90247daac1548bbb421b015'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:36:17.108810 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-60e890a6-6806-4d97-97be-d5f9a10385d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db34ffc8f90247daac1548bbb421b015'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:36:17.109584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-60e890a6-6806-4d97-97be-d5f9a10385d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db34ffc8f90247daac1548bbb421b015'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:36:17.122895 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1040/2059] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:36:16 2022] DELETE /identity/v3/users/db34ffc8f90247daac1548bbb421b015 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:36:17.297277 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76746ee-0ff3-4ad1-ad8c-facf4a64a1c6 service placement] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:36:17.310242 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76746ee-0ff3-4ad1-ad8c-facf4a64a1c6 service placement] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:36:17.369159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb69a135-fec4-489a-91e4-4c742d0844c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:36:17.369790 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69a135-fec4-489a-91e4-4c742d0844c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:36:17.370070 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69a135-fec4-489a-91e4-4c742d0844c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:36:17.370344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69a135-fec4-489a-91e4-4c742d0844c9 admin admin] PATH_INFO: `/v3/projects/66a33a727f9c40b99fb14bd2e0fc5090` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:36:17.380394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb69a135-fec4-489a-91e4-4c742d0844c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66a33a727f9c40b99fb14bd2e0fc5090)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:36:17.382588 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb69a135-fec4-489a-91e4-4c742d0844c9 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:36:17.427937 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fb69a135-fec4-489a-91e4-4c742d0844c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66a33a727f9c40b99fb14bd2e0fc5090', 'request_id': 'req-fb69a135-fec4-489a-91e4-4c742d0844c9'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:36:17.433516 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fb69a135-fec4-489a-91e4-4c742d0844c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66a33a727f9c40b99fb14bd2e0fc5090', 'request_id': 'req-fb69a135-fec4-489a-91e4-4c742d0844c9'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:36:17.440147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fb69a135-fec4-489a-91e4-4c742d0844c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66a33a727f9c40b99fb14bd2e0fc5090', 'request_id': 'req-fb69a135-fec4-489a-91e4-4c742d0844c9'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:36:17.452393 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fb69a135-fec4-489a-91e4-4c742d0844c9 admin admin] The token cache is being invalidate because project 66a33a727f9c40b99fb14bd2e0fc5090 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:36:17.452718 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fb69a135-fec4-489a-91e4-4c742d0844c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:36:17.453343 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fb69a135-fec4-489a-91e4-4c742d0844c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:36:17.454828 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1020/2060] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:36:17 2022] DELETE /identity/v3/projects/66a33a727f9c40b99fb14bd2e0fc5090 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:36:17.461342 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e6ad5-5980-4cde-97f0-2e0141172b69 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:36:17.461707 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e6ad5-5980-4cde-97f0-2e0141172b69 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:36:17.461980 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e6ad5-5980-4cde-97f0-2e0141172b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:36:17.481927 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-8d1e6ad5-5980-4cde-97f0-2e0141172b69 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:36:17.483103 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8d1e6ad5-5980-4cde-97f0-2e0141172b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:36:17.514773 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:36:17.544285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1041/2061] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:36:17.552307 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0908f9-4d1c-4eaa-aa5a-08c5fd1f622f None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:36:17.552591 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0908f9-4d1c-4eaa-aa5a-08c5fd1f622f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:36:17.552943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0908f9-4d1c-4eaa-aa5a-08c5fd1f622f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:36:17.573151 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-0a0908f9-4d1c-4eaa-aa5a-08c5fd1f622f None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:36:17.574361 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0a0908f9-4d1c-4eaa-aa5a-08c5fd1f622f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:36:17.582295 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1021/2062] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:36:17.587506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e6ad5-5980-4cde-97f0-2e0141172b69 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:36:17.597056 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e6ad5-5980-4cde-97f0-2e0141172b69 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:36:17.602103 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c526094-a883-4259-8538-b3efbc21508e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:36:17.602696 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c526094-a883-4259-8538-b3efbc21508e admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:36:17.602974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c526094-a883-4259-8538-b3efbc21508e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:36:17.603245 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c526094-a883-4259-8538-b3efbc21508e admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:36:17.603878 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c526094-a883-4259-8538-b3efbc21508e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:36:17.604178 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c526094-a883-4259-8538-b3efbc21508e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:36:17.605789 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c526094-a883-4259-8538-b3efbc21508e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:36:17.617559 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1042/2063] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:36:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:36:17.621822 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0908f9-4d1c-4eaa-aa5a-08c5fd1f622f None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:36:17.631024 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0908f9-4d1c-4eaa-aa5a-08c5fd1f622f None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:36:17.636578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43cc94-eda1-4b75-9ff8-7d915208da8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:36:17.637252 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43cc94-eda1-4b75-9ff8-7d915208da8e admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:36:17.637524 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43cc94-eda1-4b75-9ff8-7d915208da8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:36:17.637791 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43cc94-eda1-4b75-9ff8-7d915208da8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:36:17.638420 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd43cc94-eda1-4b75-9ff8-7d915208da8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:36:17.640293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd43cc94-eda1-4b75-9ff8-7d915208da8e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:36:17.664582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:36:17.677659 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1022/2064] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:36:17 2022] POST /identity/v3/projects => generated 367 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:36:17.682180 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c526094-a883-4259-8538-b3efbc21508e admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:36:17.691748 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c526094-a883-4259-8538-b3efbc21508e admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:36:17.745986 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80be0dc-6bc4-47f0-b8b7-5cda01f20bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:36:17.746580 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80be0dc-6bc4-47f0-b8b7-5cda01f20bbf admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:36:17.746853 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80be0dc-6bc4-47f0-b8b7-5cda01f20bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:36:17.747118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80be0dc-6bc4-47f0-b8b7-5cda01f20bbf admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:36:17.747737 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80be0dc-6bc4-47f0-b8b7-5cda01f20bbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:36:17.749708 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80be0dc-6bc4-47f0-b8b7-5cda01f20bbf admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:36:17.770725 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1043/2065] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:36:17 2022] POST /identity/v3/users => generated 284 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:36:17.775870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43cc94-eda1-4b75-9ff8-7d915208da8e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:36:17.786245 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43cc94-eda1-4b75-9ff8-7d915208da8e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:36:17.791078 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818c046-3d1e-4b8c-860a-c1b90e064ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:36:17.791677 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818c046-3d1e-4b8c-860a-c1b90e064ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:36:17.791953 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818c046-3d1e-4b8c-860a-c1b90e064ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:36:17.792227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818c046-3d1e-4b8c-860a-c1b90e064ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:36:17.792898 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818c046-3d1e-4b8c-860a-c1b90e064ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:36:17.794478 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818c046-3d1e-4b8c-860a-c1b90e064ef8 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:36:17.804277 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1023/2066] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:36:17.808131 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80be0dc-6bc4-47f0-b8b7-5cda01f20bbf admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:36:17.818028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80be0dc-6bc4-47f0-b8b7-5cda01f20bbf admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:36:17.823079 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb0f80-325e-44f0-8680-a4a3754493fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:36:17.823673 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb0f80-325e-44f0-8680-a4a3754493fa admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:36:17.823953 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb0f80-325e-44f0-8680-a4a3754493fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:36:17.824225 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb0f80-325e-44f0-8680-a4a3754493fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:36:17.824924 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbb0f80-325e-44f0-8680-a4a3754493fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:36:17.826614 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbb0f80-325e-44f0-8680-a4a3754493fa admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:36:17.837863 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1044/2067] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:36:17.841853 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818c046-3d1e-4b8c-860a-c1b90e064ef8 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:36:17.852858 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818c046-3d1e-4b8c-860a-c1b90e064ef8 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:36:17.857888 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948c47a-7521-489a-b6c2-582150b716cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:36:17.858588 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948c47a-7521-489a-b6c2-582150b716cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:36:17.859022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948c47a-7521-489a-b6c2-582150b716cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:36:17.859298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948c47a-7521-489a-b6c2-582150b716cc admin admin] PATH_INFO: `/v3/projects/463dde56779941e2962df7aace0c4afa/users/39a129f162224e2ab34280d1a53aad66/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:36:17.876030 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948c47a-7521-489a-b6c2-582150b716cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=463dde56779941e2962df7aace0c4afa, user_id=39a129f162224e2ab34280d1a53aad66, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:36:17.879145 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948c47a-7521-489a-b6c2-582150b716cc admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:36:17.892903 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1024/2068] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:36:17 2022] PUT /identity/v3/projects/463dde56779941e2962df7aace0c4afa/users/39a129f162224e2ab34280d1a53aad66/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:36:17.905161 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ab59b-fef0-4639-85c2-333ee07efe1d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:36:17.905448 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ab59b-fef0-4639-85c2-333ee07efe1d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:36:17.905726 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ab59b-fef0-4639-85c2-333ee07efe1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:36:17.933165 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-388ab59b-fef0-4639-85c2-333ee07efe1d None None] MFA Rules not processed for user `39a129f162224e2ab34280d1a53aad66`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:36:17.934404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-388ab59b-fef0-4639-85c2-333ee07efe1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:36:17.996668 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1045/2069] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:36:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:36:27.889035 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:INSERT=2,keystone:SELECT=17 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:36:27.993736 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=82,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:36:38.457008 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948c47a-7521-489a-b6c2-582150b716cc admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:36:38.458819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3948c47a-7521-489a-b6c2-582150b716cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:36:38.514762 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948c47a-7521-489a-b6c2-582150b716cc admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:36:38.518296 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244eb189-b4f2-4449-8efd-dd6e5cdb5676 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:36:38.518883 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244eb189-b4f2-4449-8efd-dd6e5cdb5676 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:36:38.519157 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244eb189-b4f2-4449-8efd-dd6e5cdb5676 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:36:38.519424 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244eb189-b4f2-4449-8efd-dd6e5cdb5676 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:36:38.529499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244eb189-b4f2-4449-8efd-dd6e5cdb5676 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:36:38.530955 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244eb189-b4f2-4449-8efd-dd6e5cdb5676 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:36:38.539875 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1025/2070] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:36:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:36:48.535829 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=26 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:37:48.971900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ab59b-fef0-4639-85c2-333ee07efe1d None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:48.988538 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ab59b-fef0-4639-85c2-333ee07efe1d None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:48.993211 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb8f00-3f4c-46e6-bc51-f137a1af49e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:48.995305 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb8f00-3f4c-46e6-bc51-f137a1af49e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:48.995759 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb8f00-3f4c-46e6-bc51-f137a1af49e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:48.996190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb8f00-3f4c-46e6-bc51-f137a1af49e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:48.998455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d7cb8f00-3f4c-46e6-bc51-f137a1af49e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:49.031725 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244eb189-b4f2-4449-8efd-dd6e5cdb5676 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:49.033215 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-244eb189-b4f2-4449-8efd-dd6e5cdb5676 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:49.069552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb8f00-3f4c-46e6-bc51-f137a1af49e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:49.071533 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb8f00-3f4c-46e6-bc51-f137a1af49e1 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:49.088894 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1046/2071] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:37:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:37:49.145835 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244eb189-b4f2-4449-8efd-dd6e5cdb5676 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:49.151512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744de19-b4f1-405f-8857-4db0b741b00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:49.152816 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744de19-b4f1-405f-8857-4db0b741b00b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:49.153109 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744de19-b4f1-405f-8857-4db0b741b00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:49.153525 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744de19-b4f1-405f-8857-4db0b741b00b admin admin] PATH_INFO: `/v3/users/6efdaa628f0644228d51686cc1a6bbdc` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:49.167898 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744de19-b4f1-405f-8857-4db0b741b00b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6efdaa628f0644228d51686cc1a6bbdc)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:49.186036 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744de19-b4f1-405f-8857-4db0b741b00b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:49.186145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb8f00-3f4c-46e6-bc51-f137a1af49e1 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:49.186145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d7cb8f00-3f4c-46e6-bc51-f137a1af49e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:49.196086 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb8f00-3f4c-46e6-bc51-f137a1af49e1 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:49.201261 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c358f-c21f-40a4-9fef-7bb9ec283dd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:49.202338 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c358f-c21f-40a4-9fef-7bb9ec283dd1 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:49.202857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c358f-c21f-40a4-9fef-7bb9ec283dd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:49.203505 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c358f-c21f-40a4-9fef-7bb9ec283dd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:49.206056 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-660c358f-c21f-40a4-9fef-7bb9ec283dd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:49.271010 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d744de19-b4f1-405f-8857-4db0b741b00b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6efdaa628f0644228d51686cc1a6bbdc'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:49.271799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d744de19-b4f1-405f-8857-4db0b741b00b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6efdaa628f0644228d51686cc1a6bbdc'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:49.303019 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d744de19-b4f1-405f-8857-4db0b741b00b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6efdaa628f0644228d51686cc1a6bbdc'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:49.326388 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d744de19-b4f1-405f-8857-4db0b741b00b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6efdaa628f0644228d51686cc1a6bbdc'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:49.336591 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d744de19-b4f1-405f-8857-4db0b741b00b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6efdaa628f0644228d51686cc1a6bbdc'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:49.347645 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1026/2072] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:37:49 2022] DELETE /identity/v3/users/6efdaa628f0644228d51686cc1a6bbdc => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:37:49.348754 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744de19-b4f1-405f-8857-4db0b741b00b admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:49.350273 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d744de19-b4f1-405f-8857-4db0b741b00b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:49.400604 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c358f-c21f-40a4-9fef-7bb9ec283dd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:49.402423 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c358f-c21f-40a4-9fef-7bb9ec283dd1 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:49.426823 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1047/2073] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:37:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:37:49.427897 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c358f-c21f-40a4-9fef-7bb9ec283dd1 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:49.430828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-660c358f-c21f-40a4-9fef-7bb9ec283dd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:49.457457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744de19-b4f1-405f-8857-4db0b741b00b admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:49.464188 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579d9d7-5063-45c4-b157-2114c080bd92 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dcfe5244bd04c00b6dde1d6f795d4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:49.464474 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579d9d7-5063-45c4-b157-2114c080bd92 service glance] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:49.464861 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579d9d7-5063-45c4-b157-2114c080bd92 service glance] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:49.465196 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579d9d7-5063-45c4-b157-2114c080bd92 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:49.467870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2579d9d7-5063-45c4-b157-2114c080bd92 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:49.509665 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c358f-c21f-40a4-9fef-7bb9ec283dd1 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:49.512004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-2579d9d7-5063-45c4-b157-2114c080bd92 service glance] Could not find project: 31067798d7384f4489c25dd5660b823f.: keystone.exception.ProjectNotFound: Could not find project: 31067798d7384f4489c25dd5660b823f. Oct 20 06:37:49.512004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 06:37:49.512004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 06:37:49.512004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 06:37:49.512004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 06:37:49.512004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 06:37:49.512004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 06:37:49.512004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 06:37:49.512004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 06:37:49.512004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 06:37:49.512004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 06:37:49.512004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 06:37:49.512004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Oct 20 06:37:49.512004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Oct 20 06:37:49.512004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Oct 20 06:37:49.513122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Oct 20 06:37:49.513122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Oct 20 06:37:49.513122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Oct 20 06:37:49.513122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 06:37:49.513122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 06:37:49.513122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Oct 20 06:37:49.513122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Oct 20 06:37:49.513122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 06:37:49.513122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 06:37:49.513122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 06:37:49.513122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 06:37:49.513122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 06:37:49.513122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 06:37:49.513122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 06:37:49.513122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 06:37:49.513122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 06:37:49.513122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 06:37:49.514259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 06:37:49.514259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 06:37:49.514259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 06:37:49.514259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 06:37:49.514259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Oct 20 06:37:49.514259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Oct 20 06:37:49.514259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Oct 20 06:37:49.514259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._validate_token_resources() Oct 20 06:37:49.514259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Oct 20 06:37:49.514259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Oct 20 06:37:49.514259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Oct 20 06:37:49.514259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Oct 20 06:37:49.514259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 06:37:49.514259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 06:37:49.514259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 06:37:49.514259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 06:37:49.514259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 06:37:49.515542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 06:37:49.515542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 06:37:49.515542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 20 06:37:49.515542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 06:37:49.515542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 20 06:37:49.515542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 06:37:49.515542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 06:37:49.515542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 06:37:49.515542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 20 06:37:49.515542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 06:37:49.515542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 20 06:37:49.515542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 06:37:49.515542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 06:37:49.515542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 06:37:49.515542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 06:37:49.515542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 06:37:49.515542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 06:37:49.516578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 31067798d7384f4489c25dd5660b823f. Oct 20 06:37:49.516578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 20 06:37:49.516578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1027/2074] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:37:49 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 20 06:37:49.516578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdb5c4-1aa8-4e9a-a53b-329357adf990 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:49.516578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdb5c4-1aa8-4e9a-a53b-329357adf990 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:49.516578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdb5c4-1aa8-4e9a-a53b-329357adf990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:49.523254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845958dc-956f-4ee7-801e-a042c0c3e8f6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dcfe5244bd04c00b6dde1d6f795d4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:49.523988 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845958dc-956f-4ee7-801e-a042c0c3e8f6 service glance] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:49.524436 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845958dc-956f-4ee7-801e-a042c0c3e8f6 service glance] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:49.525494 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845958dc-956f-4ee7-801e-a042c0c3e8f6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:49.528449 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-845958dc-956f-4ee7-801e-a042c0c3e8f6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:49.545785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-845958dc-956f-4ee7-801e-a042c0c3e8f6 service glance] Could not find project: 0cfa9834ec244713816c8775c3ba7f18.: keystone.exception.ProjectNotFound: Could not find project: 0cfa9834ec244713816c8775c3ba7f18. Oct 20 06:37:49.545785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 06:37:49.545785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 06:37:49.545785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 06:37:49.545785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 06:37:49.545785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 06:37:49.545785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 06:37:49.545785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 06:37:49.545785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 06:37:49.545785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 06:37:49.545785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 06:37:49.545785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 06:37:49.545785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Oct 20 06:37:49.545785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Oct 20 06:37:49.545785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Oct 20 06:37:49.552553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Oct 20 06:37:49.552553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Oct 20 06:37:49.552553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Oct 20 06:37:49.552553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 06:37:49.552553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 06:37:49.552553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Oct 20 06:37:49.552553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Oct 20 06:37:49.552553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 06:37:49.552553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 06:37:49.552553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 06:37:49.552553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 06:37:49.552553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 06:37:49.552553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 06:37:49.552553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 06:37:49.552553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 06:37:49.552553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 06:37:49.552553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 06:37:49.553584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 06:37:49.553584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 06:37:49.553584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 06:37:49.553584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 06:37:49.553584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Oct 20 06:37:49.553584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Oct 20 06:37:49.553584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Oct 20 06:37:49.553584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._validate_token_resources() Oct 20 06:37:49.553584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Oct 20 06:37:49.553584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Oct 20 06:37:49.553584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Oct 20 06:37:49.553584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Oct 20 06:37:49.553584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 06:37:49.553584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 06:37:49.553584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 06:37:49.553584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 06:37:49.553584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 06:37:49.554579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 06:37:49.554579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 06:37:49.554579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 06:37:49.554579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 06:37:49.554579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 06:37:49.554579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 06:37:49.554579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 06:37:49.554579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 06:37:49.554579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 06:37:49.554579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 06:37:49.554579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 06:37:49.554579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 06:37:49.554579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 06:37:49.554579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 06:37:49.554579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 06:37:49.554579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 06:37:49.554579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 06:37:49.555445 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 0cfa9834ec244713816c8775c3ba7f18. Oct 20 06:37:49.555445 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 06:37:49.555445 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1048/2075] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:37:49 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 120 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 20 06:37:49.555625 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:37:49.584203 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3dbdb5c4-1aa8-4e9a-a53b-329357adf990 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:37:49.587161 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3dbdb5c4-1aa8-4e9a-a53b-329357adf990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:49.589006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845958dc-956f-4ee7-801e-a042c0c3e8f6 service glance] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:49.600062 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845958dc-956f-4ee7-801e-a042c0c3e8f6 service glance] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:49.605691 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031015d0-2333-4786-ad78-d31324f99b37 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dcfe5244bd04c00b6dde1d6f795d4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:49.606456 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031015d0-2333-4786-ad78-d31324f99b37 service glance] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:49.606811 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031015d0-2333-4786-ad78-d31324f99b37 service glance] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:49.607156 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031015d0-2333-4786-ad78-d31324f99b37 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:49.610815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-031015d0-2333-4786-ad78-d31324f99b37 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:49.632057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-031015d0-2333-4786-ad78-d31324f99b37 service glance] Could not find project: db144186ebfa49c185816e5c986b6098.: keystone.exception.ProjectNotFound: Could not find project: db144186ebfa49c185816e5c986b6098. Oct 20 06:37:49.632057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 06:37:49.632057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 06:37:49.632057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 06:37:49.632057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 06:37:49.632057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 06:37:49.632057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 06:37:49.632057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 06:37:49.632057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 06:37:49.632057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 06:37:49.632057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 06:37:49.632057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 06:37:49.632057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Oct 20 06:37:49.632057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Oct 20 06:37:49.632057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Oct 20 06:37:49.633157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Oct 20 06:37:49.633157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Oct 20 06:37:49.633157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Oct 20 06:37:49.633157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 06:37:49.633157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 06:37:49.633157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Oct 20 06:37:49.633157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Oct 20 06:37:49.633157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 06:37:49.633157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 06:37:49.633157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 06:37:49.633157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 06:37:49.633157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 06:37:49.633157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 06:37:49.633157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 06:37:49.633157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 06:37:49.633157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 06:37:49.633157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 06:37:49.634177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 06:37:49.634177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 06:37:49.634177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 06:37:49.634177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 06:37:49.634177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Oct 20 06:37:49.634177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Oct 20 06:37:49.634177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Oct 20 06:37:49.634177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._validate_token_resources() Oct 20 06:37:49.634177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Oct 20 06:37:49.634177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Oct 20 06:37:49.634177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Oct 20 06:37:49.634177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Oct 20 06:37:49.634177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 06:37:49.634177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 06:37:49.634177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 06:37:49.634177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 06:37:49.634177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 06:37:49.636228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 06:37:49.636228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 06:37:49.636228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 20 06:37:49.636228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 06:37:49.636228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 20 06:37:49.636228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 06:37:49.636228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 06:37:49.636228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 06:37:49.636228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 20 06:37:49.636228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 06:37:49.636228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 20 06:37:49.636228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 06:37:49.636228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 06:37:49.636228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 06:37:49.636228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 06:37:49.636228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 06:37:49.636228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 06:37:49.637364 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: db144186ebfa49c185816e5c986b6098. Oct 20 06:37:49.637364 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 20 06:37:49.637850 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1049/2076] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:37:49 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 50 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Oct 20 06:37:49.679828 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1028/2077] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:37:49.690875 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031015d0-2333-4786-ad78-d31324f99b37 service glance] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:49.692368 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-031015d0-2333-4786-ad78-d31324f99b37 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:49.697557 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:37:49.786652 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031015d0-2333-4786-ad78-d31324f99b37 service glance] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:49.792616 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b013c1-d0b3-4a41-a8d3-dad772d3852a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:49.793620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b013c1-d0b3-4a41-a8d3-dad772d3852a service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:49.794045 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b013c1-d0b3-4a41-a8d3-dad772d3852a service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:49.794512 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b013c1-d0b3-4a41-a8d3-dad772d3852a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:49.810376 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b013c1-d0b3-4a41-a8d3-dad772d3852a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:49.811977 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b013c1-d0b3-4a41-a8d3-dad772d3852a service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:49.832994 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1050/2078] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:37:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:37:49.902189 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdb5c4-1aa8-4e9a-a53b-329357adf990 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:49.903912 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3dbdb5c4-1aa8-4e9a-a53b-329357adf990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:49.920352 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdb5c4-1aa8-4e9a-a53b-329357adf990 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:49.926417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600f8bb-ba47-477a-85b8-88df699661b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:49.926799 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600f8bb-ba47-477a-85b8-88df699661b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:49.927162 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600f8bb-ba47-477a-85b8-88df699661b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:49.927434 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600f8bb-ba47-477a-85b8-88df699661b1 admin admin] PATH_INFO: `/v3/users/ec0cf867400c4ec3b880f5258ed9b0fd` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:49.986175 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600f8bb-ba47-477a-85b8-88df699661b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec0cf867400c4ec3b880f5258ed9b0fd)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:49.992255 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600f8bb-ba47-477a-85b8-88df699661b1 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:50.086115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5600f8bb-ba47-477a-85b8-88df699661b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec0cf867400c4ec3b880f5258ed9b0fd'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:50.087275 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5600f8bb-ba47-477a-85b8-88df699661b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec0cf867400c4ec3b880f5258ed9b0fd'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:50.112085 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5600f8bb-ba47-477a-85b8-88df699661b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec0cf867400c4ec3b880f5258ed9b0fd'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:50.129288 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5600f8bb-ba47-477a-85b8-88df699661b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec0cf867400c4ec3b880f5258ed9b0fd'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:50.141229 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5600f8bb-ba47-477a-85b8-88df699661b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec0cf867400c4ec3b880f5258ed9b0fd'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:50.148390 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1029/2079] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:37:49 2022] DELETE /identity/v3/users/ec0cf867400c4ec3b880f5258ed9b0fd => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:37:50.153262 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8b80e-8ea2-442b-9987-d3597b95a6ac None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:50.153965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8b80e-8ea2-442b-9987-d3597b95a6ac None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:50.153965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8b80e-8ea2-442b-9987-d3597b95a6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:50.178019 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-c7e8b80e-8ea2-442b-9987-d3597b95a6ac None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:37:50.179520 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c7e8b80e-8ea2-442b-9987-d3597b95a6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:50.204136 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600f8bb-ba47-477a-85b8-88df699661b1 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:50.208036 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5600f8bb-ba47-477a-85b8-88df699661b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:50.282040 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1051/2080] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:37:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:37:50.289567 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8b80e-8ea2-442b-9987-d3597b95a6ac None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:50.293452 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c7e8b80e-8ea2-442b-9987-d3597b95a6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:50.360727 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600f8bb-ba47-477a-85b8-88df699661b1 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:50.366250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de02e16-0f81-4a25-98e0-4414622af586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:50.366997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de02e16-0f81-4a25-98e0-4414622af586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:50.367358 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de02e16-0f81-4a25-98e0-4414622af586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:50.367820 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de02e16-0f81-4a25-98e0-4414622af586 admin admin] PATH_INFO: `/v3/projects/87f5b76358024015a9bf1d9a49a4f222` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:50.369177 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de02e16-0f81-4a25-98e0-4414622af586 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87f5b76358024015a9bf1d9a49a4f222)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:50.371448 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de02e16-0f81-4a25-98e0-4414622af586 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:50.374200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:37:50.380076 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8b80e-8ea2-442b-9987-d3597b95a6ac None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:50.385793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ae7e4-4709-4764-b0ae-3abfa70e6c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:50.386940 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ae7e4-4709-4764-b0ae-3abfa70e6c88 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:50.387630 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ae7e4-4709-4764-b0ae-3abfa70e6c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:50.387630 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ae7e4-4709-4764-b0ae-3abfa70e6c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:50.397699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ae7e4-4709-4764-b0ae-3abfa70e6c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:50.399264 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ae7e4-4709-4764-b0ae-3abfa70e6c88 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:50.411641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1052/2081] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:37:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:37:50.441628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7de02e16-0f81-4a25-98e0-4414622af586 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87f5b76358024015a9bf1d9a49a4f222', 'request_id': 'req-7de02e16-0f81-4a25-98e0-4414622af586'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:50.450596 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7de02e16-0f81-4a25-98e0-4414622af586 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87f5b76358024015a9bf1d9a49a4f222', 'request_id': 'req-7de02e16-0f81-4a25-98e0-4414622af586'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:50.461321 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7de02e16-0f81-4a25-98e0-4414622af586 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87f5b76358024015a9bf1d9a49a4f222', 'request_id': 'req-7de02e16-0f81-4a25-98e0-4414622af586'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:50.479359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7de02e16-0f81-4a25-98e0-4414622af586 admin admin] The token cache is being invalidate because project 87f5b76358024015a9bf1d9a49a4f222 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:37:50.479965 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7de02e16-0f81-4a25-98e0-4414622af586 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:50.481127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7de02e16-0f81-4a25-98e0-4414622af586 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:50.483026 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1030/2082] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:37:50 2022] DELETE /identity/v3/projects/87f5b76358024015a9bf1d9a49a4f222 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:37:50.489461 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d5061-0cf4-4f4c-9885-c5e928c3d6f0 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:50.489814 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d5061-0cf4-4f4c-9885-c5e928c3d6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:50.490289 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d5061-0cf4-4f4c-9885-c5e928c3d6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:50.563688 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-8a2d5061-0cf4-4f4c-9885-c5e928c3d6f0 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:37:50.565748 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8a2d5061-0cf4-4f4c-9885-c5e928c3d6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:50.579352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:37:50.698617 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1053/2083] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:37:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:37:50.707681 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833e07b-14ac-4fb7-8603-a10daba7dfaf None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:50.707985 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833e07b-14ac-4fb7-8603-a10daba7dfaf None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:50.708251 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833e07b-14ac-4fb7-8603-a10daba7dfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:50.729104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-7833e07b-14ac-4fb7-8603-a10daba7dfaf None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:37:50.730321 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7833e07b-14ac-4fb7-8603-a10daba7dfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:50.738817 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1031/2084] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:37:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:37:50.742999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d5061-0cf4-4f4c-9885-c5e928c3d6f0 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:50.755784 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d5061-0cf4-4f4c-9885-c5e928c3d6f0 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:50.761166 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af24b9-5cff-493a-b341-20148f166266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:50.761800 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af24b9-5cff-493a-b341-20148f166266 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:50.762173 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af24b9-5cff-493a-b341-20148f166266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:50.762474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af24b9-5cff-493a-b341-20148f166266 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:50.763123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af24b9-5cff-493a-b341-20148f166266 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:37:50.763449 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af24b9-5cff-493a-b341-20148f166266 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:50.765942 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af24b9-5cff-493a-b341-20148f166266 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:50.771882 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833e07b-14ac-4fb7-8603-a10daba7dfaf None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:50.773464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7833e07b-14ac-4fb7-8603-a10daba7dfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:50.782506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1054/2085] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:37:50.786592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af24b9-5cff-493a-b341-20148f166266 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:50.793049 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833e07b-14ac-4fb7-8603-a10daba7dfaf None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:50.797451 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af24b9-5cff-493a-b341-20148f166266 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:50.799647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c9b56-a652-4d4d-883f-29fe06a7b535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:50.800290 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c9b56-a652-4d4d-883f-29fe06a7b535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:50.800574 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c9b56-a652-4d4d-883f-29fe06a7b535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:50.800888 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c9b56-a652-4d4d-883f-29fe06a7b535 admin admin] PATH_INFO: `/v3/projects/8fdac492af4040c4957ca72f0849657f` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:50.802871 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50059e06-1996-4f6d-8344-7646e06a3896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:50.803467 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50059e06-1996-4f6d-8344-7646e06a3896 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:50.803739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50059e06-1996-4f6d-8344-7646e06a3896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:50.804147 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50059e06-1996-4f6d-8344-7646e06a3896 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:50.804815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50059e06-1996-4f6d-8344-7646e06a3896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:50.807585 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50059e06-1996-4f6d-8344-7646e06a3896 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:50.813077 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c9b56-a652-4d4d-883f-29fe06a7b535 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fdac492af4040c4957ca72f0849657f)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:50.815002 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c9b56-a652-4d4d-883f-29fe06a7b535 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:50.839376 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1055/2086] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:37:50 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:37:50.843483 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50059e06-1996-4f6d-8344-7646e06a3896 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:50.866171 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-319c9b56-a652-4d4d-883f-29fe06a7b535 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fdac492af4040c4957ca72f0849657f', 'request_id': 'req-319c9b56-a652-4d4d-883f-29fe06a7b535'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:50.872164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-319c9b56-a652-4d4d-883f-29fe06a7b535 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fdac492af4040c4957ca72f0849657f', 'request_id': 'req-319c9b56-a652-4d4d-883f-29fe06a7b535'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:50.879116 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-319c9b56-a652-4d4d-883f-29fe06a7b535 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fdac492af4040c4957ca72f0849657f', 'request_id': 'req-319c9b56-a652-4d4d-883f-29fe06a7b535'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:50.898304 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-319c9b56-a652-4d4d-883f-29fe06a7b535 admin admin] The token cache is being invalidate because project 8fdac492af4040c4957ca72f0849657f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:37:50.898603 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-319c9b56-a652-4d4d-883f-29fe06a7b535 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:50.899259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-319c9b56-a652-4d4d-883f-29fe06a7b535 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:50.904942 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1032/2087] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:37:50 2022] DELETE /identity/v3/projects/8fdac492af4040c4957ca72f0849657f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:37:50.905944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c9b56-a652-4d4d-883f-29fe06a7b535 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:50.907706 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-319c9b56-a652-4d4d-883f-29fe06a7b535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:50.913932 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50059e06-1996-4f6d-8344-7646e06a3896 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:50.976605 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acf95a-63c1-48e5-a883-4840bf885100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:50.977245 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acf95a-63c1-48e5-a883-4840bf885100 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:50.977518 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acf95a-63c1-48e5-a883-4840bf885100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:50.977797 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acf95a-63c1-48e5-a883-4840bf885100 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:50.978479 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5acf95a-63c1-48e5-a883-4840bf885100 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:50.980386 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5acf95a-63c1-48e5-a883-4840bf885100 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:50.981357 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c9b56-a652-4d4d-883f-29fe06a7b535 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:50.986020 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9bbae-5465-4c98-9a18-e8d82a57360f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:50.986647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9bbae-5465-4c98-9a18-e8d82a57360f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:50.986924 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9bbae-5465-4c98-9a18-e8d82a57360f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:50.987196 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9bbae-5465-4c98-9a18-e8d82a57360f admin admin] PATH_INFO: `/v3/users/6a25a8cd8ff046f0ad04320a3c50d93a` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:50.988324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9bbae-5465-4c98-9a18-e8d82a57360f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a25a8cd8ff046f0ad04320a3c50d93a)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:50.990267 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9bbae-5465-4c98-9a18-e8d82a57360f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:50.991823 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:37:51.007887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1056/2088] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:37:50 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:37:51.008887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acf95a-63c1-48e5-a883-4840bf885100 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:51.010651 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c5acf95a-63c1-48e5-a883-4840bf885100 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:51.022521 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acf95a-63c1-48e5-a883-4840bf885100 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:51.028806 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17a8c6-6b99-4b8a-bfbb-4b3c3a9888dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:51.029456 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17a8c6-6b99-4b8a-bfbb-4b3c3a9888dd admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:51.029760 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17a8c6-6b99-4b8a-bfbb-4b3c3a9888dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:51.030028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17a8c6-6b99-4b8a-bfbb-4b3c3a9888dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:51.030667 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17a8c6-6b99-4b8a-bfbb-4b3c3a9888dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:51.032315 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17a8c6-6b99-4b8a-bfbb-4b3c3a9888dd admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:51.042976 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1057/2089] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:37:51.046840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17a8c6-6b99-4b8a-bfbb-4b3c3a9888dd admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:51.046967 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0ce9bbae-5465-4c98-9a18-e8d82a57360f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a25a8cd8ff046f0ad04320a3c50d93a'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:51.047602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0ce9bbae-5465-4c98-9a18-e8d82a57360f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a25a8cd8ff046f0ad04320a3c50d93a'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:51.054892 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:37:51.055449 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0ce9bbae-5465-4c98-9a18-e8d82a57360f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a25a8cd8ff046f0ad04320a3c50d93a'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:51.059534 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17a8c6-6b99-4b8a-bfbb-4b3c3a9888dd admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:51.065269 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b9fb7-c88e-4ec1-84ba-b93f23a59075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:51.065871 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b9fb7-c88e-4ec1-84ba-b93f23a59075 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:51.066144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b9fb7-c88e-4ec1-84ba-b93f23a59075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:51.066442 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b9fb7-c88e-4ec1-84ba-b93f23a59075 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:51.067211 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b9fb7-c88e-4ec1-84ba-b93f23a59075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:51.068719 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0ce9bbae-5465-4c98-9a18-e8d82a57360f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a25a8cd8ff046f0ad04320a3c50d93a'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:51.068845 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b9fb7-c88e-4ec1-84ba-b93f23a59075 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:51.076534 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0ce9bbae-5465-4c98-9a18-e8d82a57360f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a25a8cd8ff046f0ad04320a3c50d93a'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:51.079713 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1058/2090] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:37:51.082545 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1033/2091] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:37:50 2022] DELETE /identity/v3/users/6a25a8cd8ff046f0ad04320a3c50d93a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:37:51.083517 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b9fb7-c88e-4ec1-84ba-b93f23a59075 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:51.085242 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ed6b9fb7-c88e-4ec1-84ba-b93f23a59075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:51.098141 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b9fb7-c88e-4ec1-84ba-b93f23a59075 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:51.158068 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db408439-b05d-41d8-9edd-04ad0afb49ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:51.158965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db408439-b05d-41d8-9edd-04ad0afb49ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:51.158965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db408439-b05d-41d8-9edd-04ad0afb49ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:51.159231 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db408439-b05d-41d8-9edd-04ad0afb49ef admin admin] PATH_INFO: `/v3/projects/c6d602fe085a4d7d82788e61287b352d/users/30be578ddd664bc79498d9a21f708f04/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:51.175509 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db408439-b05d-41d8-9edd-04ad0afb49ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6d602fe085a4d7d82788e61287b352d, user_id=30be578ddd664bc79498d9a21f708f04, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:51.178727 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db408439-b05d-41d8-9edd-04ad0afb49ef admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:51.194705 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1059/2092] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:37:51 2022] PUT /identity/v3/projects/c6d602fe085a4d7d82788e61287b352d/users/30be578ddd664bc79498d9a21f708f04/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:37:51.200602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e3550-e673-464c-808a-60d03c7cb7d3 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:51.201198 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e3550-e673-464c-808a-60d03c7cb7d3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:51.201458 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e3550-e673-464c-808a-60d03c7cb7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:51.222641 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-234e3550-e673-464c-808a-60d03c7cb7d3 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:37:51.223861 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-234e3550-e673-464c-808a-60d03c7cb7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:51.290438 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1034/2093] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:37:51.305145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db408439-b05d-41d8-9edd-04ad0afb49ef admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:51.307323 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-db408439-b05d-41d8-9edd-04ad0afb49ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:51.374361 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e3550-e673-464c-808a-60d03c7cb7d3 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:51.376487 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-234e3550-e673-464c-808a-60d03c7cb7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:51.388007 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db408439-b05d-41d8-9edd-04ad0afb49ef admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:51.392211 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8863b7f-f1c4-4ba1-b4a4-eb5232f7324f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:51.392946 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8863b7f-f1c4-4ba1-b4a4-eb5232f7324f service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:51.393227 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8863b7f-f1c4-4ba1-b4a4-eb5232f7324f service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:51.393498 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8863b7f-f1c4-4ba1-b4a4-eb5232f7324f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:51.402449 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e3550-e673-464c-808a-60d03c7cb7d3 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:51.403869 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8863b7f-f1c4-4ba1-b4a4-eb5232f7324f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:51.405346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8863b7f-f1c4-4ba1-b4a4-eb5232f7324f service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:51.407604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c812b84-6e14-47d3-8c79-4204c4d694d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:51.408991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c812b84-6e14-47d3-8c79-4204c4d694d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:51.409614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c812b84-6e14-47d3-8c79-4204c4d694d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:51.409935 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c812b84-6e14-47d3-8c79-4204c4d694d9 admin admin] PATH_INFO: `/v3/projects/90e61099d1854a62b3d797a0c77e94f0` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:51.411213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c812b84-6e14-47d3-8c79-4204c4d694d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90e61099d1854a62b3d797a0c77e94f0)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:51.413073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c812b84-6e14-47d3-8c79-4204c4d694d9 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:51.418210 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1060/2094] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:37:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:37:51.467422 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2c812b84-6e14-47d3-8c79-4204c4d694d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90e61099d1854a62b3d797a0c77e94f0', 'request_id': 'req-2c812b84-6e14-47d3-8c79-4204c4d694d9'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:51.473225 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2c812b84-6e14-47d3-8c79-4204c4d694d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90e61099d1854a62b3d797a0c77e94f0', 'request_id': 'req-2c812b84-6e14-47d3-8c79-4204c4d694d9'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:51.480378 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2c812b84-6e14-47d3-8c79-4204c4d694d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90e61099d1854a62b3d797a0c77e94f0', 'request_id': 'req-2c812b84-6e14-47d3-8c79-4204c4d694d9'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:51.495957 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2c812b84-6e14-47d3-8c79-4204c4d694d9 admin admin] The token cache is being invalidate because project 90e61099d1854a62b3d797a0c77e94f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:37:51.496311 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2c812b84-6e14-47d3-8c79-4204c4d694d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:51.497031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2c812b84-6e14-47d3-8c79-4204c4d694d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:51.498792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1035/2095] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:37:51 2022] DELETE /identity/v3/projects/90e61099d1854a62b3d797a0c77e94f0 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:37:51.503809 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686dd26a-640a-433e-8b15-49349c835c48 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:51.504092 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686dd26a-640a-433e-8b15-49349c835c48 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:51.504368 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686dd26a-640a-433e-8b15-49349c835c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:51.527182 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-686dd26a-640a-433e-8b15-49349c835c48 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:37:51.528406 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-686dd26a-640a-433e-8b15-49349c835c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:51.593032 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1061/2096] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:37:51.601021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3ca52-4743-4d70-a992-1ed2de34a2c9 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:51.601253 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3ca52-4743-4d70-a992-1ed2de34a2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:51.601571 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3ca52-4743-4d70-a992-1ed2de34a2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:51.609624 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:37:51.625929 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-39d3ca52-4743-4d70-a992-1ed2de34a2c9 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:37:51.627166 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-39d3ca52-4743-4d70-a992-1ed2de34a2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:51.635588 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1036/2097] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:37:51.639857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686dd26a-640a-433e-8b15-49349c835c48 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:51.649225 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686dd26a-640a-433e-8b15-49349c835c48 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:51.654352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bfa96a-a928-444b-a0e6-fd8059c108b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:51.654946 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfa96a-a928-444b-a0e6-fd8059c108b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:51.655217 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfa96a-a928-444b-a0e6-fd8059c108b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:51.655485 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfa96a-a928-444b-a0e6-fd8059c108b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:51.656123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bfa96a-a928-444b-a0e6-fd8059c108b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:37:51.656422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bfa96a-a928-444b-a0e6-fd8059c108b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:51.658042 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bfa96a-a928-444b-a0e6-fd8059c108b6 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:51.673431 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1062/2098] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:37:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:37:51.674541 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3ca52-4743-4d70-a992-1ed2de34a2c9 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:51.684312 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3ca52-4743-4d70-a992-1ed2de34a2c9 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:51.694548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c86b2-14b7-4ad2-8d8e-2dceaef138fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:51.695205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c86b2-14b7-4ad2-8d8e-2dceaef138fd admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:51.695538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c86b2-14b7-4ad2-8d8e-2dceaef138fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:51.695835 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c86b2-14b7-4ad2-8d8e-2dceaef138fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:51.696492 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c86b2-14b7-4ad2-8d8e-2dceaef138fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:51.699238 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c86b2-14b7-4ad2-8d8e-2dceaef138fd admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:51.730375 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1037/2099] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:37:51 2022] POST /identity/v3/projects => generated 393 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:37:51.734660 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bfa96a-a928-444b-a0e6-fd8059c108b6 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:51.749210 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bfa96a-a928-444b-a0e6-fd8059c108b6 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:51.754106 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:37:51.823353 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ca706-68d5-4d60-a109-0075c5394d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:51.823974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ca706-68d5-4d60-a109-0075c5394d5f admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:51.824232 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ca706-68d5-4d60-a109-0075c5394d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:51.824499 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ca706-68d5-4d60-a109-0075c5394d5f admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:51.825157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ca706-68d5-4d60-a109-0075c5394d5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:51.827120 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ca706-68d5-4d60-a109-0075c5394d5f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:51.850370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1063/2100] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:37:51 2022] POST /identity/v3/users => generated 297 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:37:51.855407 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c86b2-14b7-4ad2-8d8e-2dceaef138fd admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:51.866310 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c86b2-14b7-4ad2-8d8e-2dceaef138fd admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:51.871247 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a75e49-0213-4faa-95a9-5ad595ceeb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:51.871865 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a75e49-0213-4faa-95a9-5ad595ceeb8c admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:51.872142 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a75e49-0213-4faa-95a9-5ad595ceeb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:51.872440 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a75e49-0213-4faa-95a9-5ad595ceeb8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:51.873109 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a75e49-0213-4faa-95a9-5ad595ceeb8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:51.874690 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a75e49-0213-4faa-95a9-5ad595ceeb8c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:51.886396 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1038/2101] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:37:51.890180 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ca706-68d5-4d60-a109-0075c5394d5f admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:51.900810 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ca706-68d5-4d60-a109-0075c5394d5f admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:51.905824 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2714838-5f8b-4aa8-8286-af3cb99e1bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:51.906416 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2714838-5f8b-4aa8-8286-af3cb99e1bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:51.906688 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2714838-5f8b-4aa8-8286-af3cb99e1bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:51.906953 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2714838-5f8b-4aa8-8286-af3cb99e1bf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:51.907583 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2714838-5f8b-4aa8-8286-af3cb99e1bf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:51.909193 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2714838-5f8b-4aa8-8286-af3cb99e1bf2 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:51.919794 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1064/2102] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:37:51.924035 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a75e49-0213-4faa-95a9-5ad595ceeb8c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:51.934412 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a75e49-0213-4faa-95a9-5ad595ceeb8c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:51.939629 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d786dd-a99e-434b-b896-efcf0586b192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:51.940316 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d786dd-a99e-434b-b896-efcf0586b192 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:51.940564 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d786dd-a99e-434b-b896-efcf0586b192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:51.940894 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d786dd-a99e-434b-b896-efcf0586b192 admin admin] PATH_INFO: `/v3/projects/0362a4621f9f49d8ab92211cd91ab59b/users/24b3c85ff03f46c2afbc541bbeca32c8/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:51.956716 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d786dd-a99e-434b-b896-efcf0586b192 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0362a4621f9f49d8ab92211cd91ab59b, user_id=24b3c85ff03f46c2afbc541bbeca32c8, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:51.959426 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d786dd-a99e-434b-b896-efcf0586b192 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:51.975456 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1039/2103] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:37:51 2022] PUT /identity/v3/projects/0362a4621f9f49d8ab92211cd91ab59b/users/24b3c85ff03f46c2afbc541bbeca32c8/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:37:51.985970 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c59ae-cd97-4ea6-a396-1cc10d43e4a7 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:51.986302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c59ae-cd97-4ea6-a396-1cc10d43e4a7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:51.986581 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c59ae-cd97-4ea6-a396-1cc10d43e4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:52.019357 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-7f0c59ae-cd97-4ea6-a396-1cc10d43e4a7 None None] MFA Rules not processed for user `24b3c85ff03f46c2afbc541bbeca32c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:37:52.020628 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7f0c59ae-cd97-4ea6-a396-1cc10d43e4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:52.103775 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1065/2104] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:37:51 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:37:52.108455 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d786dd-a99e-434b-b896-efcf0586b192 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:52.183984 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d786dd-a99e-434b-b896-efcf0586b192 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:52.190478 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2109d9-35c9-44df-86d1-44179af47550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:52.191222 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2109d9-35c9-44df-86d1-44179af47550 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:52.192552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2109d9-35c9-44df-86d1-44179af47550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:52.192877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2109d9-35c9-44df-86d1-44179af47550 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:52.193518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2109d9-35c9-44df-86d1-44179af47550 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:52.196385 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2109d9-35c9-44df-86d1-44179af47550 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:52.231378 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1040/2105] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:37:52 2022] POST /identity/v3/projects => generated 391 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:37:52.232050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c59ae-cd97-4ea6-a396-1cc10d43e4a7 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:52.242657 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c59ae-cd97-4ea6-a396-1cc10d43e4a7 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:52.317941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d178726-7036-4b7b-9efa-6bf7ba170b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:52.319375 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d178726-7036-4b7b-9efa-6bf7ba170b39 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:52.320106 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d178726-7036-4b7b-9efa-6bf7ba170b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:52.320611 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d178726-7036-4b7b-9efa-6bf7ba170b39 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:52.321643 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d178726-7036-4b7b-9efa-6bf7ba170b39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:52.324177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d178726-7036-4b7b-9efa-6bf7ba170b39 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:52.352010 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1066/2106] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:37:52 2022] POST /identity/v3/users => generated 302 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:37:52.356608 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2109d9-35c9-44df-86d1-44179af47550 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:52.372537 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2109d9-35c9-44df-86d1-44179af47550 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:52.380318 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb80a7-3a6e-4d91-b932-13e63e37d0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:52.382213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb80a7-3a6e-4d91-b932-13e63e37d0bd admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:52.382606 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb80a7-3a6e-4d91-b932-13e63e37d0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:52.382987 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb80a7-3a6e-4d91-b932-13e63e37d0bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:52.384293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb80a7-3a6e-4d91-b932-13e63e37d0bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:52.386512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb80a7-3a6e-4d91-b932-13e63e37d0bd admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:52.409173 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1041/2107] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:37:52.412944 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d178726-7036-4b7b-9efa-6bf7ba170b39 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:52.421290 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:37:52.426079 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d178726-7036-4b7b-9efa-6bf7ba170b39 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:52.434337 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786398e-5673-4fee-b599-928726f957d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:52.435475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786398e-5673-4fee-b599-928726f957d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:52.435901 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786398e-5673-4fee-b599-928726f957d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:52.436406 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786398e-5673-4fee-b599-928726f957d5 admin admin] PATH_INFO: `/v3/projects/afb632d48bb448f99cf0e0d6141962f9/users/5b558015a379482c9843b79cee68abe6/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:52.459457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4786398e-5673-4fee-b599-928726f957d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afb632d48bb448f99cf0e0d6141962f9, user_id=5b558015a379482c9843b79cee68abe6, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:52.463730 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4786398e-5673-4fee-b599-928726f957d5 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:52.488918 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1067/2108] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:37:52 2022] PUT /identity/v3/projects/afb632d48bb448f99cf0e0d6141962f9/users/5b558015a379482c9843b79cee68abe6/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:37:52.492818 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb80a7-3a6e-4d91-b932-13e63e37d0bd admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:52.516054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb80a7-3a6e-4d91-b932-13e63e37d0bd admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:52.589098 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed603d8-c9d2-4aff-9786-b8d92f6d6b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:52.590211 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed603d8-c9d2-4aff-9786-b8d92f6d6b89 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:52.590667 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed603d8-c9d2-4aff-9786-b8d92f6d6b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:52.591223 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed603d8-c9d2-4aff-9786-b8d92f6d6b89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:52.592038 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed603d8-c9d2-4aff-9786-b8d92f6d6b89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:52.594226 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed603d8-c9d2-4aff-9786-b8d92f6d6b89 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:52.612531 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1042/2109] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:37:52.613667 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786398e-5673-4fee-b599-928726f957d5 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:52.625478 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786398e-5673-4fee-b599-928726f957d5 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:52.631206 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063be44b-1fa2-453e-b8e3-12abac242bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:52.632026 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063be44b-1fa2-453e-b8e3-12abac242bfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:52.632390 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063be44b-1fa2-453e-b8e3-12abac242bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:52.632799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063be44b-1fa2-453e-b8e3-12abac242bfe admin admin] PATH_INFO: `/v3/projects/afb632d48bb448f99cf0e0d6141962f9/users/5b558015a379482c9843b79cee68abe6/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:52.635956 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063be44b-1fa2-453e-b8e3-12abac242bfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=afb632d48bb448f99cf0e0d6141962f9, user_id=5b558015a379482c9843b79cee68abe6, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:52.638864 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063be44b-1fa2-453e-b8e3-12abac242bfe admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:52.651043 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1068/2110] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:37:52 2022] PUT /identity/v3/projects/afb632d48bb448f99cf0e0d6141962f9/users/5b558015a379482c9843b79cee68abe6/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:37:52.723229 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c4055-20b1-4b87-afe1-6e8f9cee724c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:52.723666 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c4055-20b1-4b87-afe1-6e8f9cee724c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:52.724013 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c4055-20b1-4b87-afe1-6e8f9cee724c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:52.751928 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:37:52.765132 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-e66c4055-20b1-4b87-afe1-6e8f9cee724c None None] MFA Rules not processed for user `5b558015a379482c9843b79cee68abe6`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:37:52.766929 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e66c4055-20b1-4b87-afe1-6e8f9cee724c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:52.865528 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1043/2111] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:37:52 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:37:52.882665 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063be44b-1fa2-453e-b8e3-12abac242bfe admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:52.884458 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-063be44b-1fa2-453e-b8e3-12abac242bfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:52.949593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063be44b-1fa2-453e-b8e3-12abac242bfe admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:52.953526 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac4e36-b336-4e0b-ae5c-26959757e33d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:52.954152 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac4e36-b336-4e0b-ae5c-26959757e33d service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:52.954433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac4e36-b336-4e0b-ae5c-26959757e33d service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:52.954770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac4e36-b336-4e0b-ae5c-26959757e33d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:52.965130 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac4e36-b336-4e0b-ae5c-26959757e33d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:52.966899 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac4e36-b336-4e0b-ae5c-26959757e33d service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:52.988461 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1069/2112] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:37:52 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:37:53.084956 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c4055-20b1-4b87-afe1-6e8f9cee724c None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:53.099132 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c4055-20b1-4b87-afe1-6e8f9cee724c None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:53.103609 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ad469-16f9-4a43-aa27-c3e664b43785 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:53.104645 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ad469-16f9-4a43-aa27-c3e664b43785 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:53.105020 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ad469-16f9-4a43-aa27-c3e664b43785 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:53.106744 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ad469-16f9-4a43-aa27-c3e664b43785 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:53.178673 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ad469-16f9-4a43-aa27-c3e664b43785 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:53.180213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ad469-16f9-4a43-aa27-c3e664b43785 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:53.196129 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1044/2113] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:37:53 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:37:56.367002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd4f3a-7ecd-46f0-855c-2bb8e2819eda None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:56.367490 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd4f3a-7ecd-46f0-855c-2bb8e2819eda None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:56.367673 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd4f3a-7ecd-46f0-855c-2bb8e2819eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:56.438553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-59bd4f3a-7ecd-46f0-855c-2bb8e2819eda None None] MFA Rules not processed for user `30be578ddd664bc79498d9a21f708f04`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:37:56.439753 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-59bd4f3a-7ecd-46f0-855c-2bb8e2819eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:56.523055 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1070/2114] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:37:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:37:56.528458 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ad469-16f9-4a43-aa27-c3e664b43785 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:56.530935 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-555ad469-16f9-4a43-aa27-c3e664b43785 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:56.631122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ad469-16f9-4a43-aa27-c3e664b43785 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:56.638326 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddeb8be-d832-47f0-8e5e-e9e5e72c5d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:56.639715 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddeb8be-d832-47f0-8e5e-e9e5e72c5d1b admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:56.640069 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddeb8be-d832-47f0-8e5e-e9e5e72c5d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:56.640351 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddeb8be-d832-47f0-8e5e-e9e5e72c5d1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:56.641532 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddeb8be-d832-47f0-8e5e-e9e5e72c5d1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:56.644402 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddeb8be-d832-47f0-8e5e-e9e5e72c5d1b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:56.688857 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1045/2115] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:37:56 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:37:56.690346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd4f3a-7ecd-46f0-855c-2bb8e2819eda None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:56.705302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd4f3a-7ecd-46f0-855c-2bb8e2819eda None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:56.793086 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75550f-6de4-4446-9441-9fffa2c30c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:56.794037 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75550f-6de4-4446-9441-9fffa2c30c0f admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:56.794327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75550f-6de4-4446-9441-9fffa2c30c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:56.794604 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75550f-6de4-4446-9441-9fffa2c30c0f admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:56.795309 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e75550f-6de4-4446-9441-9fffa2c30c0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:56.797891 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e75550f-6de4-4446-9441-9fffa2c30c0f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:56.807473 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:37:56.826511 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1071/2116] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:37:56 2022] POST /identity/v3/users => generated 299 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:37:56.830450 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddeb8be-d832-47f0-8e5e-e9e5e72c5d1b admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:56.842482 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddeb8be-d832-47f0-8e5e-e9e5e72c5d1b admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:56.847960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ba28b-21df-4e69-b6a4-e8fa41c893a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:56.848567 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ba28b-21df-4e69-b6a4-e8fa41c893a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:56.848884 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ba28b-21df-4e69-b6a4-e8fa41c893a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:56.849164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ba28b-21df-4e69-b6a4-e8fa41c893a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:56.849817 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ba28b-21df-4e69-b6a4-e8fa41c893a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:56.851435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ba28b-21df-4e69-b6a4-e8fa41c893a9 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:56.864958 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1046/2117] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:37:56.870624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75550f-6de4-4446-9441-9fffa2c30c0f admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:56.898093 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75550f-6de4-4446-9441-9fffa2c30c0f admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:56.923107 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc3003-35dc-44de-be91-90100cc53d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:56.923800 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc3003-35dc-44de-be91-90100cc53d97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:56.924072 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc3003-35dc-44de-be91-90100cc53d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:56.924434 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc3003-35dc-44de-be91-90100cc53d97 admin admin] PATH_INFO: `/v3/projects/3a42dd67974d4409b4c1401e6f965e3d/users/5fd3f793b3f04519bdb213f237d7d7ca/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:56.960467 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc3003-35dc-44de-be91-90100cc53d97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a42dd67974d4409b4c1401e6f965e3d, user_id=5fd3f793b3f04519bdb213f237d7d7ca, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:56.963946 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc3003-35dc-44de-be91-90100cc53d97 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:56.983687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1072/2118] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:37:56 2022] PUT /identity/v3/projects/3a42dd67974d4409b4c1401e6f965e3d/users/5fd3f793b3f04519bdb213f237d7d7ca/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:37:56.987919 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ba28b-21df-4e69-b6a4-e8fa41c893a9 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:57.031878 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ba28b-21df-4e69-b6a4-e8fa41c893a9 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:57.036660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:37:57.125428 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72668810-8235-4920-b86b-6016b30dc32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:57.125428 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72668810-8235-4920-b86b-6016b30dc32e admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:57.125428 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72668810-8235-4920-b86b-6016b30dc32e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:57.125428 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72668810-8235-4920-b86b-6016b30dc32e admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:57.125741 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72668810-8235-4920-b86b-6016b30dc32e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:57.125741 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72668810-8235-4920-b86b-6016b30dc32e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:57.127138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc3003-35dc-44de-be91-90100cc53d97 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:57.138475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc3003-35dc-44de-be91-90100cc53d97 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:57.140743 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1047/2119] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:37:57.144566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72668810-8235-4920-b86b-6016b30dc32e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:57.146416 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e1ee8-a1f8-4295-9cc1-d8e794ea2be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:57.147149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e1ee8-a1f8-4295-9cc1-d8e794ea2be7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:57.147491 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e1ee8-a1f8-4295-9cc1-d8e794ea2be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:57.147843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e1ee8-a1f8-4295-9cc1-d8e794ea2be7 admin admin] PATH_INFO: `/v3/users/24b3c85ff03f46c2afbc541bbeca32c8` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:57.149167 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e1ee8-a1f8-4295-9cc1-d8e794ea2be7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24b3c85ff03f46c2afbc541bbeca32c8)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:57.151430 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e1ee8-a1f8-4295-9cc1-d8e794ea2be7 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:57.155232 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72668810-8235-4920-b86b-6016b30dc32e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:57.161617 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5a52b-bc41-49d2-9261-e67e6c773621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:57.162301 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5a52b-bc41-49d2-9261-e67e6c773621 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:57.162577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5a52b-bc41-49d2-9261-e67e6c773621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:57.162850 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5a52b-bc41-49d2-9261-e67e6c773621 admin admin] PATH_INFO: `/v3/projects/3a42dd67974d4409b4c1401e6f965e3d/users/5fd3f793b3f04519bdb213f237d7d7ca/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:57.165036 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e5a52b-bc41-49d2-9261-e67e6c773621 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a42dd67974d4409b4c1401e6f965e3d, user_id=5fd3f793b3f04519bdb213f237d7d7ca, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:57.168357 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e5a52b-bc41-49d2-9261-e67e6c773621 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:57.181895 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1048/2120] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:37:57 2022] PUT /identity/v3/projects/3a42dd67974d4409b4c1401e6f965e3d/users/5fd3f793b3f04519bdb213f237d7d7ca/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:37:57.224823 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a25e1ee8-a1f8-4295-9cc1-d8e794ea2be7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24b3c85ff03f46c2afbc541bbeca32c8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:57.226062 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a25e1ee8-a1f8-4295-9cc1-d8e794ea2be7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24b3c85ff03f46c2afbc541bbeca32c8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:57.237385 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a25e1ee8-a1f8-4295-9cc1-d8e794ea2be7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24b3c85ff03f46c2afbc541bbeca32c8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:57.255728 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a25e1ee8-a1f8-4295-9cc1-d8e794ea2be7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24b3c85ff03f46c2afbc541bbeca32c8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:57.256660 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a25e1ee8-a1f8-4295-9cc1-d8e794ea2be7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24b3c85ff03f46c2afbc541bbeca32c8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:57.272032 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1073/2121] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:37:57 2022] DELETE /identity/v3/users/24b3c85ff03f46c2afbc541bbeca32c8 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:37:57.276514 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fadebd-1973-47fd-99f5-4254edf31592 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:57.276870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fadebd-1973-47fd-99f5-4254edf31592 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:57.277105 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fadebd-1973-47fd-99f5-4254edf31592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:57.301551 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-e2fadebd-1973-47fd-99f5-4254edf31592 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:37:57.302753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e2fadebd-1973-47fd-99f5-4254edf31592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:57.384558 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1049/2122] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:37:57.403969 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e1ee8-a1f8-4295-9cc1-d8e794ea2be7 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:57.405584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a25e1ee8-a1f8-4295-9cc1-d8e794ea2be7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:57.473774 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e1ee8-a1f8-4295-9cc1-d8e794ea2be7 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:57.478509 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974836c-47d1-45c0-8b9f-681c4981f3c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:57.479363 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974836c-47d1-45c0-8b9f-681c4981f3c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:57.479642 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974836c-47d1-45c0-8b9f-681c4981f3c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:57.479912 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974836c-47d1-45c0-8b9f-681c4981f3c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:57.491100 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974836c-47d1-45c0-8b9f-681c4981f3c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:57.492642 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974836c-47d1-45c0-8b9f-681c4981f3c6 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:57.503760 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1074/2123] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:37:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:37:57.952585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fadebd-1973-47fd-99f5-4254edf31592 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:57.962430 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e2fadebd-1973-47fd-99f5-4254edf31592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:57.990352 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fadebd-1973-47fd-99f5-4254edf31592 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:58.000413 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c2a93-cb77-4275-bcaf-feb4c1049922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:58.002210 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c2a93-cb77-4275-bcaf-feb4c1049922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:58.002538 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c2a93-cb77-4275-bcaf-feb4c1049922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:58.002813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c2a93-cb77-4275-bcaf-feb4c1049922 admin admin] PATH_INFO: `/v3/projects/0362a4621f9f49d8ab92211cd91ab59b` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:58.005808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c2a93-cb77-4275-bcaf-feb4c1049922 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0362a4621f9f49d8ab92211cd91ab59b)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:58.017002 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c2a93-cb77-4275-bcaf-feb4c1049922 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:58.092622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-236c2a93-cb77-4275-bcaf-feb4c1049922 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0362a4621f9f49d8ab92211cd91ab59b', 'request_id': 'req-236c2a93-cb77-4275-bcaf-feb4c1049922'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:58.110927 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-236c2a93-cb77-4275-bcaf-feb4c1049922 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0362a4621f9f49d8ab92211cd91ab59b', 'request_id': 'req-236c2a93-cb77-4275-bcaf-feb4c1049922'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:58.120422 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-236c2a93-cb77-4275-bcaf-feb4c1049922 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0362a4621f9f49d8ab92211cd91ab59b', 'request_id': 'req-236c2a93-cb77-4275-bcaf-feb4c1049922'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:58.140032 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-236c2a93-cb77-4275-bcaf-feb4c1049922 admin admin] The token cache is being invalidate because project 0362a4621f9f49d8ab92211cd91ab59b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:37:58.140674 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-236c2a93-cb77-4275-bcaf-feb4c1049922 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:58.141505 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-236c2a93-cb77-4275-bcaf-feb4c1049922 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:58.146581 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1050/2124] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:37:57 2022] DELETE /identity/v3/projects/0362a4621f9f49d8ab92211cd91ab59b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:37:58.148338 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974836c-47d1-45c0-8b9f-681c4981f3c6 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:58.162901 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974836c-47d1-45c0-8b9f-681c4981f3c6 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:58.254361 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:INSERT=5,keystone:SELECT=85,keystone:DELETE=10 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:37:58.286924 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374985b6-d250-4df3-9636-677812702f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:58.287626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374985b6-d250-4df3-9636-677812702f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:58.287875 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374985b6-d250-4df3-9636-677812702f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:58.288287 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374985b6-d250-4df3-9636-677812702f86 admin admin] PATH_INFO: `/v3/users/5b558015a379482c9843b79cee68abe6` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:58.289570 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374985b6-d250-4df3-9636-677812702f86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b558015a379482c9843b79cee68abe6)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:58.292564 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374985b6-d250-4df3-9636-677812702f86 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:58.361479 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-374985b6-d250-4df3-9636-677812702f86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b558015a379482c9843b79cee68abe6'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:58.362298 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-374985b6-d250-4df3-9636-677812702f86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b558015a379482c9843b79cee68abe6'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:58.373320 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-374985b6-d250-4df3-9636-677812702f86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b558015a379482c9843b79cee68abe6'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:58.390751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-374985b6-d250-4df3-9636-677812702f86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b558015a379482c9843b79cee68abe6'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:58.391746 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-374985b6-d250-4df3-9636-677812702f86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b558015a379482c9843b79cee68abe6'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:58.408237 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1075/2125] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:37:58 2022] DELETE /identity/v3/users/5b558015a379482c9843b79cee68abe6 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:37:58.796878 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c2a93-cb77-4275-bcaf-feb4c1049922 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:58.799939 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-236c2a93-cb77-4275-bcaf-feb4c1049922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:58.828215 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c2a93-cb77-4275-bcaf-feb4c1049922 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:58.846505 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:37:58.994865 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766f011-a311-4923-9abb-6e3da7027a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:58.996187 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766f011-a311-4923-9abb-6e3da7027a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:58.996803 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766f011-a311-4923-9abb-6e3da7027a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:58.997265 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766f011-a311-4923-9abb-6e3da7027a41 admin admin] PATH_INFO: `/v3/projects/afb632d48bb448f99cf0e0d6141962f9` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:58.998972 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e766f011-a311-4923-9abb-6e3da7027a41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afb632d48bb448f99cf0e0d6141962f9)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:59.001170 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e766f011-a311-4923-9abb-6e3da7027a41 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:59.144405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e766f011-a311-4923-9abb-6e3da7027a41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afb632d48bb448f99cf0e0d6141962f9', 'request_id': 'req-e766f011-a311-4923-9abb-6e3da7027a41'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:59.187231 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e766f011-a311-4923-9abb-6e3da7027a41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afb632d48bb448f99cf0e0d6141962f9', 'request_id': 'req-e766f011-a311-4923-9abb-6e3da7027a41'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:59.198729 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e766f011-a311-4923-9abb-6e3da7027a41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afb632d48bb448f99cf0e0d6141962f9', 'request_id': 'req-e766f011-a311-4923-9abb-6e3da7027a41'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:59.225612 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e766f011-a311-4923-9abb-6e3da7027a41 admin admin] The token cache is being invalidate because project afb632d48bb448f99cf0e0d6141962f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:37:59.226280 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e766f011-a311-4923-9abb-6e3da7027a41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:59.228558 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e766f011-a311-4923-9abb-6e3da7027a41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:37:59.231833 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1051/2126] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:37:58 2022] DELETE /identity/v3/projects/afb632d48bb448f99cf0e0d6141962f9 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:37:59.238912 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f106a5f-05b3-4e7d-a69d-88af7965d68b None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:59.240999 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f106a5f-05b3-4e7d-a69d-88af7965d68b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:59.241304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f106a5f-05b3-4e7d-a69d-88af7965d68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:59.289458 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-7f106a5f-05b3-4e7d-a69d-88af7965d68b None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:37:59.292028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7f106a5f-05b3-4e7d-a69d-88af7965d68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:59.445971 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1076/2127] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:37:59.449812 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f3203-9de6-4bea-88c4-63ae1343e858 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:59.450103 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f3203-9de6-4bea-88c4-63ae1343e858 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:59.450377 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f3203-9de6-4bea-88c4-63ae1343e858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:59.483395 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-317f3203-9de6-4bea-88c4-63ae1343e858 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:37:59.485003 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-317f3203-9de6-4bea-88c4-63ae1343e858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:37:59.507012 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f106a5f-05b3-4e7d-a69d-88af7965d68b None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:59.507240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1052/2128] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:37:59.518604 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f106a5f-05b3-4e7d-a69d-88af7965d68b None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:59.525569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d44127-80da-48d7-9129-6c88e3bf7c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:59.526290 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d44127-80da-48d7-9129-6c88e3bf7c9f admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:59.526576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d44127-80da-48d7-9129-6c88e3bf7c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:59.526994 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d44127-80da-48d7-9129-6c88e3bf7c9f admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:59.527770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d44127-80da-48d7-9129-6c88e3bf7c9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:37:59.528200 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d44127-80da-48d7-9129-6c88e3bf7c9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:59.530030 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d44127-80da-48d7-9129-6c88e3bf7c9f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:59.548297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1077/2129] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:37:59.551537 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f3203-9de6-4bea-88c4-63ae1343e858 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:59.563571 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f3203-9de6-4bea-88c4-63ae1343e858 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:59.568395 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0d295-ed61-4a68-aeda-026e2e513471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:59.569034 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0d295-ed61-4a68-aeda-026e2e513471 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:59.569318 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0d295-ed61-4a68-aeda-026e2e513471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:59.569595 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0d295-ed61-4a68-aeda-026e2e513471 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:59.570354 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f0d295-ed61-4a68-aeda-026e2e513471 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:59.572304 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f0d295-ed61-4a68-aeda-026e2e513471 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:59.604782 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1053/2130] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:37:59 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:37:59.608862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d44127-80da-48d7-9129-6c88e3bf7c9f admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:59.620438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d44127-80da-48d7-9129-6c88e3bf7c9f admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:59.678432 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:37:59.708479 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3feb026-6b5d-4e66-aa9d-ca06cbddf235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:59.710767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3feb026-6b5d-4e66-aa9d-ca06cbddf235 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:59.711253 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3feb026-6b5d-4e66-aa9d-ca06cbddf235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:59.711669 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3feb026-6b5d-4e66-aa9d-ca06cbddf235 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:59.713309 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3feb026-6b5d-4e66-aa9d-ca06cbddf235 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:59.715449 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3feb026-6b5d-4e66-aa9d-ca06cbddf235 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:59.758697 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0d295-ed61-4a68-aeda-026e2e513471 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:59.768840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1078/2131] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:37:59 2022] POST /identity/v3/users => generated 291 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:37:59.773347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0d295-ed61-4a68-aeda-026e2e513471 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:59.779382 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06b40b-80a4-4a45-bada-9974adb5c450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:59.780033 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06b40b-80a4-4a45-bada-9974adb5c450 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:59.780311 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06b40b-80a4-4a45-bada-9974adb5c450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:59.780578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06b40b-80a4-4a45-bada-9974adb5c450 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:59.781281 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06b40b-80a4-4a45-bada-9974adb5c450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:59.783275 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06b40b-80a4-4a45-bada-9974adb5c450 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:59.795203 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1054/2132] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:37:59.799552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3feb026-6b5d-4e66-aa9d-ca06cbddf235 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:59.814233 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3feb026-6b5d-4e66-aa9d-ca06cbddf235 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:59.827287 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa0b714-755a-48e4-8eda-e12b7794bd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:59.828154 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0b714-755a-48e4-8eda-e12b7794bd36 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:59.828439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0b714-755a-48e4-8eda-e12b7794bd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:59.828733 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0b714-755a-48e4-8eda-e12b7794bd36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:59.829923 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa0b714-755a-48e4-8eda-e12b7794bd36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:59.832360 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa0b714-755a-48e4-8eda-e12b7794bd36 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:59.855014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06b40b-80a4-4a45-bada-9974adb5c450 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:37:59.862473 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1079/2133] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:37:59.867060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06b40b-80a4-4a45-bada-9974adb5c450 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:37:59.874639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf8786-37c9-4c34-bfa5-d0c2fcdad6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:37:59.875453 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf8786-37c9-4c34-bfa5-d0c2fcdad6ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:59.876193 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf8786-37c9-4c34-bfa5-d0c2fcdad6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:59.876552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf8786-37c9-4c34-bfa5-d0c2fcdad6ab admin admin] PATH_INFO: `/v3/projects/8b7d88f039644b71b2552a3c3083a70c/users/370a0bde5b3d4526a2eefd5a52fd8a38/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:59.898172 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf8786-37c9-4c34-bfa5-d0c2fcdad6ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b7d88f039644b71b2552a3c3083a70c, user_id=370a0bde5b3d4526a2eefd5a52fd8a38, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:37:59.902100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf8786-37c9-4c34-bfa5-d0c2fcdad6ab admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:37:59.933494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1055/2134] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:37:59 2022] PUT /identity/v3/projects/8b7d88f039644b71b2552a3c3083a70c/users/370a0bde5b3d4526a2eefd5a52fd8a38/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:37:59.944867 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652b886-a1ac-4888-9a0e-54cbae35c81b None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:37:59.945421 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652b886-a1ac-4888-9a0e-54cbae35c81b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:37:59.946141 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652b886-a1ac-4888-9a0e-54cbae35c81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:37:59.983169 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d652b886-a1ac-4888-9a0e-54cbae35c81b None None] MFA Rules not processed for user `370a0bde5b3d4526a2eefd5a52fd8a38`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:37:59.984506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d652b886-a1ac-4888-9a0e-54cbae35c81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:00.149263 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1080/2135] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:37:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:00.153998 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf8786-37c9-4c34-bfa5-d0c2fcdad6ab admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:00.269671 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:00.328516 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf8786-37c9-4c34-bfa5-d0c2fcdad6ab admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:00.347125 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b43b06-cfa3-4685-9cc3-ebf4a25faf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:00.347125 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b43b06-cfa3-4685-9cc3-ebf4a25faf1c admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:00.347125 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b43b06-cfa3-4685-9cc3-ebf4a25faf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:00.347125 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b43b06-cfa3-4685-9cc3-ebf4a25faf1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:00.347466 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b43b06-cfa3-4685-9cc3-ebf4a25faf1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:00.347466 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b43b06-cfa3-4685-9cc3-ebf4a25faf1c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:00.414674 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1056/2136] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:38:00 2022] POST /identity/v3/projects => generated 381 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:00.419771 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652b886-a1ac-4888-9a0e-54cbae35c81b None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:00.438614 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652b886-a1ac-4888-9a0e-54cbae35c81b None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:00.577469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b67f82-61c1-4b0a-b4c0-b31aa3cf04b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:00.577469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b67f82-61c1-4b0a-b4c0-b31aa3cf04b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:00.577469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b67f82-61c1-4b0a-b4c0-b31aa3cf04b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:00.577469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b67f82-61c1-4b0a-b4c0-b31aa3cf04b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:00.578318 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b67f82-61c1-4b0a-b4c0-b31aa3cf04b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:00.579144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b67f82-61c1-4b0a-b4c0-b31aa3cf04b1 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:00.580234 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b43b06-cfa3-4685-9cc3-ebf4a25faf1c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:00.582200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e6b43b06-cfa3-4685-9cc3-ebf4a25faf1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:00.607774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b43b06-cfa3-4685-9cc3-ebf4a25faf1c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:00.619073 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1081/2137] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:38:00 2022] POST /identity/v3/users => generated 297 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:00.623136 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b67f82-61c1-4b0a-b4c0-b31aa3cf04b1 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:00.629260 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f614000-60e3-4253-8b1f-36264d988138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:00.631161 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f614000-60e3-4253-8b1f-36264d988138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:00.631650 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f614000-60e3-4253-8b1f-36264d988138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:00.632174 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f614000-60e3-4253-8b1f-36264d988138 admin admin] PATH_INFO: `/v3/users/39a129f162224e2ab34280d1a53aad66` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:00.635011 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f614000-60e3-4253-8b1f-36264d988138 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39a129f162224e2ab34280d1a53aad66)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:00.640583 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f614000-60e3-4253-8b1f-36264d988138 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:00.651221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b67f82-61c1-4b0a-b4c0-b31aa3cf04b1 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:00.667075 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa3dd55-433f-4899-a19a-4e9f97e7ccc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:00.668272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa3dd55-433f-4899-a19a-4e9f97e7ccc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:00.668916 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa3dd55-433f-4899-a19a-4e9f97e7ccc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:00.669419 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa3dd55-433f-4899-a19a-4e9f97e7ccc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:00.681380 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa3dd55-433f-4899-a19a-4e9f97e7ccc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:00.683671 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa3dd55-433f-4899-a19a-4e9f97e7ccc4 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:00.698899 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1082/2138] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:38:00.702858 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa3dd55-433f-4899-a19a-4e9f97e7ccc4 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:00.729502 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa3dd55-433f-4899-a19a-4e9f97e7ccc4 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:00.745282 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b466b-b375-4133-8a22-3d3431268ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:00.753339 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b466b-b375-4133-8a22-3d3431268ab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:00.755802 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1f614000-60e3-4253-8b1f-36264d988138 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39a129f162224e2ab34280d1a53aad66'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:00.757851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1f614000-60e3-4253-8b1f-36264d988138 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39a129f162224e2ab34280d1a53aad66'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:00.764146 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b466b-b375-4133-8a22-3d3431268ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:00.764612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b466b-b375-4133-8a22-3d3431268ab5 admin admin] PATH_INFO: `/v3/projects/e864b27e497c479ea4080a650a2fde9b/users/8e6c7ebb29bf4c46adebfb56db366d6f/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:00.778402 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1f614000-60e3-4253-8b1f-36264d988138 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39a129f162224e2ab34280d1a53aad66'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:00.801659 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b466b-b375-4133-8a22-3d3431268ab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e864b27e497c479ea4080a650a2fde9b, user_id=8e6c7ebb29bf4c46adebfb56db366d6f, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:00.806617 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b466b-b375-4133-8a22-3d3431268ab5 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:00.815824 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1f614000-60e3-4253-8b1f-36264d988138 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39a129f162224e2ab34280d1a53aad66'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:00.835266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1083/2139] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:38:00 2022] PUT /identity/v3/projects/e864b27e497c479ea4080a650a2fde9b/users/8e6c7ebb29bf4c46adebfb56db366d6f/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:38:00.840380 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b466b-b375-4133-8a22-3d3431268ab5 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:00.842720 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1f614000-60e3-4253-8b1f-36264d988138 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39a129f162224e2ab34280d1a53aad66'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:00.851241 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1057/2140] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:38:00 2022] DELETE /identity/v3/users/39a129f162224e2ab34280d1a53aad66 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:00.856032 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ba451-606b-4453-bd39-35e631eb6ca5 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:00.857393 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ba451-606b-4453-bd39-35e631eb6ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:00.859743 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:00.860156 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ba451-606b-4453-bd39-35e631eb6ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:00.948382 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-e09ba451-606b-4453-bd39-35e631eb6ca5 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:00.950372 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e09ba451-606b-4453-bd39-35e631eb6ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:01.026414 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b466b-b375-4133-8a22-3d3431268ab5 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:01.131388 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1058/2141] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:01.206306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ba451-606b-4453-bd39-35e631eb6ca5 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:01.207871 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e09ba451-606b-4453-bd39-35e631eb6ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:01.229732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf0dc9-2f84-4ef6-95fe-a44d70da73c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:01.232449 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf0dc9-2f84-4ef6-95fe-a44d70da73c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:01.232754 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf0dc9-2f84-4ef6-95fe-a44d70da73c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:01.233135 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf0dc9-2f84-4ef6-95fe-a44d70da73c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:01.241082 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf0dc9-2f84-4ef6-95fe-a44d70da73c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:01.247239 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf0dc9-2f84-4ef6-95fe-a44d70da73c3 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:01.283965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1084/2142] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:38:01.294580 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf0dc9-2f84-4ef6-95fe-a44d70da73c3 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:01.300101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-0faf0dc9-2f84-4ef6-95fe-a44d70da73c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:01.355428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf0dc9-2f84-4ef6-95fe-a44d70da73c3 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:01.363463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:01.375762 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126d919-d2e8-4883-9c15-1159331b200d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:01.376443 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126d919-d2e8-4883-9c15-1159331b200d admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:01.376733 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126d919-d2e8-4883-9c15-1159331b200d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:01.377016 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126d919-d2e8-4883-9c15-1159331b200d admin admin] PATH_INFO: `/v3/projects/e864b27e497c479ea4080a650a2fde9b/users/8e6c7ebb29bf4c46adebfb56db366d6f/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:01.381422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126d919-d2e8-4883-9c15-1159331b200d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e864b27e497c479ea4080a650a2fde9b, user_id=8e6c7ebb29bf4c46adebfb56db366d6f, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:01.389450 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126d919-d2e8-4883-9c15-1159331b200d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:01.406554 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ba451-606b-4453-bd39-35e631eb6ca5 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:01.419146 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1085/2143] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:38:01 2022] PUT /identity/v3/projects/e864b27e497c479ea4080a650a2fde9b/users/8e6c7ebb29bf4c46adebfb56db366d6f/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:01.429768 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf2430-128d-44c0-bf4c-96c94fc5569a None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:01.430136 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf2430-128d-44c0-bf4c-96c94fc5569a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:01.430329 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf2430-128d-44c0-bf4c-96c94fc5569a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:01.495692 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d359a-1278-4894-a6e4-1521b91da739 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:01.496373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d359a-1278-4894-a6e4-1521b91da739 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:01.496653 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d359a-1278-4894-a6e4-1521b91da739 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:01.497005 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d359a-1278-4894-a6e4-1521b91da739 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:01.549172 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-90bf2430-128d-44c0-bf4c-96c94fc5569a None None] MFA Rules not processed for user `8e6c7ebb29bf4c46adebfb56db366d6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:01.554672 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-90bf2430-128d-44c0-bf4c-96c94fc5569a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:01.678678 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d359a-1278-4894-a6e4-1521b91da739 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:01.681615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d359a-1278-4894-a6e4-1521b91da739 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:01.699235 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1059/2144] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:38:01.766054 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1086/2145] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:01 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:01.783020 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d359a-1278-4894-a6e4-1521b91da739 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:01.785532 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3f0d359a-1278-4894-a6e4-1521b91da739 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:01.913318 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d359a-1278-4894-a6e4-1521b91da739 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:01.924713 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db92c37-5a27-4870-9427-2d0c0df19668 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:01.925505 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db92c37-5a27-4870-9427-2d0c0df19668 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:01.926914 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db92c37-5a27-4870-9427-2d0c0df19668 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:01.927283 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db92c37-5a27-4870-9427-2d0c0df19668 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:01.980200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db92c37-5a27-4870-9427-2d0c0df19668 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:01.983809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db92c37-5a27-4870-9427-2d0c0df19668 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:02.002774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1060/2146] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:01 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:38:02.367797 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf2430-128d-44c0-bf4c-96c94fc5569a None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:02.369974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-90bf2430-128d-44c0-bf4c-96c94fc5569a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:02.421987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf2430-128d-44c0-bf4c-96c94fc5569a None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:02.448410 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b2410-cfc4-4983-bced-3695ce137bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:02.451516 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b2410-cfc4-4983-bced-3695ce137bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:02.452494 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b2410-cfc4-4983-bced-3695ce137bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:02.452803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b2410-cfc4-4983-bced-3695ce137bcc admin admin] PATH_INFO: `/v3/projects/463dde56779941e2962df7aace0c4afa` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:02.459409 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b2410-cfc4-4983-bced-3695ce137bcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=463dde56779941e2962df7aace0c4afa)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:02.461394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b2410-cfc4-4983-bced-3695ce137bcc admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:02.550343 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:02.601976 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-013b2410-cfc4-4983-bced-3695ce137bcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '463dde56779941e2962df7aace0c4afa', 'request_id': 'req-013b2410-cfc4-4983-bced-3695ce137bcc'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:02.609300 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-013b2410-cfc4-4983-bced-3695ce137bcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '463dde56779941e2962df7aace0c4afa', 'request_id': 'req-013b2410-cfc4-4983-bced-3695ce137bcc'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:02.616421 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-013b2410-cfc4-4983-bced-3695ce137bcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '463dde56779941e2962df7aace0c4afa', 'request_id': 'req-013b2410-cfc4-4983-bced-3695ce137bcc'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:02.637426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-013b2410-cfc4-4983-bced-3695ce137bcc admin admin] The token cache is being invalidate because project 463dde56779941e2962df7aace0c4afa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:38:02.637740 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-013b2410-cfc4-4983-bced-3695ce137bcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:02.638776 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-013b2410-cfc4-4983-bced-3695ce137bcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:02.641006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1087/2147] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:38:02 2022] DELETE /identity/v3/projects/463dde56779941e2962df7aace0c4afa => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:02.646454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8f828-7f7f-4e06-9111-253be37c5927 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:02.647286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8f828-7f7f-4e06-9111-253be37c5927 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:02.647755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8f828-7f7f-4e06-9111-253be37c5927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:02.716237 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-73e8f828-7f7f-4e06-9111-253be37c5927 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:02.717588 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-73e8f828-7f7f-4e06-9111-253be37c5927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:02.752513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b2410-cfc4-4983-bced-3695ce137bcc admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:02.754216 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-013b2410-cfc4-4983-bced-3695ce137bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:02.803388 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:02.846509 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b2410-cfc4-4983-bced-3695ce137bcc admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:02.850749 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1061/2148] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:02.852448 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970af014-5263-44df-b4f6-cec2efb5d2dd None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:02.852448 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970af014-5263-44df-b4f6-cec2efb5d2dd None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:02.853570 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970af014-5263-44df-b4f6-cec2efb5d2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:02.857341 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6fe74d-f4cf-4cc5-9f85-470022a10475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:02.858506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6fe74d-f4cf-4cc5-9f85-470022a10475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:02.858990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6fe74d-f4cf-4cc5-9f85-470022a10475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:02.859588 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6fe74d-f4cf-4cc5-9f85-470022a10475 admin admin] PATH_INFO: `/v3/users/370a0bde5b3d4526a2eefd5a52fd8a38` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:02.863575 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6fe74d-f4cf-4cc5-9f85-470022a10475 admin admin] RBAC: Authorizing `identity:delete_user(user_id=370a0bde5b3d4526a2eefd5a52fd8a38)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:02.866360 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6fe74d-f4cf-4cc5-9f85-470022a10475 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:02.883994 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-970af014-5263-44df-b4f6-cec2efb5d2dd None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:02.886517 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-970af014-5263-44df-b4f6-cec2efb5d2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:02.903631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1062/2149] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:02.904755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970af014-5263-44df-b4f6-cec2efb5d2dd None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:02.931916 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970af014-5263-44df-b4f6-cec2efb5d2dd None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:02.942608 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28678d79-40e7-409f-9bc8-c7a314b8098a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:02.943340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28678d79-40e7-409f-9bc8-c7a314b8098a admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:02.944194 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28678d79-40e7-409f-9bc8-c7a314b8098a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:02.944991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28678d79-40e7-409f-9bc8-c7a314b8098a admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:02.945578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28678d79-40e7-409f-9bc8-c7a314b8098a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:38:02.945880 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28678d79-40e7-409f-9bc8-c7a314b8098a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:02.948651 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28678d79-40e7-409f-9bc8-c7a314b8098a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:02.963648 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-4d6fe74d-f4cf-4cc5-9f85-470022a10475 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '370a0bde5b3d4526a2eefd5a52fd8a38'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:02.965797 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-4d6fe74d-f4cf-4cc5-9f85-470022a10475 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '370a0bde5b3d4526a2eefd5a52fd8a38'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:02.969984 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1063/2150] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:38:02.970818 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28678d79-40e7-409f-9bc8-c7a314b8098a admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:02.973684 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-28678d79-40e7-409f-9bc8-c7a314b8098a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:02.982487 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-4d6fe74d-f4cf-4cc5-9f85-470022a10475 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '370a0bde5b3d4526a2eefd5a52fd8a38'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:03.006619 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-4d6fe74d-f4cf-4cc5-9f85-470022a10475 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '370a0bde5b3d4526a2eefd5a52fd8a38'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:03.008113 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-4d6fe74d-f4cf-4cc5-9f85-470022a10475 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '370a0bde5b3d4526a2eefd5a52fd8a38'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:03.016289 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28678d79-40e7-409f-9bc8-c7a314b8098a admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:03.030236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42aea20-cce5-4605-af24-f02e6fc06c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:03.032443 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42aea20-cce5-4605-af24-f02e6fc06c2b admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:03.033847 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42aea20-cce5-4605-af24-f02e6fc06c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:03.034231 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42aea20-cce5-4605-af24-f02e6fc06c2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:03.036395 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42aea20-cce5-4605-af24-f02e6fc06c2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:03.042890 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42aea20-cce5-4605-af24-f02e6fc06c2b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:03.046851 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1088/2151] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:38:02 2022] DELETE /identity/v3/users/370a0bde5b3d4526a2eefd5a52fd8a38 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:03.055263 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd27722-8310-439c-9401-e3b19098b4ce None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:03.057090 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd27722-8310-439c-9401-e3b19098b4ce None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:03.057383 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd27722-8310-439c-9401-e3b19098b4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:03.095107 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1064/2152] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:38:02 2022] POST /identity/v3/projects => generated 405 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:03.095379 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42aea20-cce5-4605-af24-f02e6fc06c2b admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:03.104830 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-bcd27722-8310-439c-9401-e3b19098b4ce None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:03.106463 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-bcd27722-8310-439c-9401-e3b19098b4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:03.151410 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:03.235794 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42aea20-cce5-4605-af24-f02e6fc06c2b admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:03.240325 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c995ec0-11ea-4024-aaf2-24fab87a616c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:03.240970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c995ec0-11ea-4024-aaf2-24fab87a616c admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:03.241252 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c995ec0-11ea-4024-aaf2-24fab87a616c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:03.241526 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c995ec0-11ea-4024-aaf2-24fab87a616c admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:03.242155 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c995ec0-11ea-4024-aaf2-24fab87a616c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:03.245277 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c995ec0-11ea-4024-aaf2-24fab87a616c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:03.277357 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1065/2153] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:38:03 2022] POST /identity/v3/users => generated 303 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:03.281636 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c995ec0-11ea-4024-aaf2-24fab87a616c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:03.295425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1089/2154] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:03.334110 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd27722-8310-439c-9401-e3b19098b4ce None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:03.335819 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-bcd27722-8310-439c-9401-e3b19098b4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:03.336012 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c995ec0-11ea-4024-aaf2-24fab87a616c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:03.343208 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732746d-e922-4995-8c63-0e56779abde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:03.343813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732746d-e922-4995-8c63-0e56779abde6 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:03.344096 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732746d-e922-4995-8c63-0e56779abde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:03.344367 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732746d-e922-4995-8c63-0e56779abde6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:03.345110 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732746d-e922-4995-8c63-0e56779abde6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:03.348012 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732746d-e922-4995-8c63-0e56779abde6 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:03.369724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1066/2155] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:38:03.375064 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732746d-e922-4995-8c63-0e56779abde6 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:03.391422 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732746d-e922-4995-8c63-0e56779abde6 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:03.410148 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ab7e8-5b1e-4755-83a1-ee57e93b7710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:03.410776 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ab7e8-5b1e-4755-83a1-ee57e93b7710 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:03.411053 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ab7e8-5b1e-4755-83a1-ee57e93b7710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:03.411758 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ab7e8-5b1e-4755-83a1-ee57e93b7710 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:03.411953 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93ab7e8-5b1e-4755-83a1-ee57e93b7710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:03.416507 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93ab7e8-5b1e-4755-83a1-ee57e93b7710 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:03.448026 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1067/2156] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:38:03.451098 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ab7e8-5b1e-4755-83a1-ee57e93b7710 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:03.480030 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ab7e8-5b1e-4755-83a1-ee57e93b7710 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:03.495085 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb9b9e-7a30-419c-b86b-2e61f4b0e9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:03.496113 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb9b9e-7a30-419c-b86b-2e61f4b0e9a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:03.496391 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb9b9e-7a30-419c-b86b-2e61f4b0e9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:03.496657 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb9b9e-7a30-419c-b86b-2e61f4b0e9a2 admin admin] PATH_INFO: `/v3/projects/528b6c5135bd4cc0bdf783383aafc468/users/b6642e04f2e14619a454b3e9eed5a62a/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:03.498675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd27722-8310-439c-9401-e3b19098b4ce None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:03.506364 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7ad4b-cb1f-420a-9069-ba214c1a0cdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:03.506989 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7ad4b-cb1f-420a-9069-ba214c1a0cdf service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:03.507262 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7ad4b-cb1f-420a-9069-ba214c1a0cdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:03.507529 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7ad4b-cb1f-420a-9069-ba214c1a0cdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:03.534365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba7ad4b-cb1f-420a-9069-ba214c1a0cdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:03.535744 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb9b9e-7a30-419c-b86b-2e61f4b0e9a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=528b6c5135bd4cc0bdf783383aafc468, user_id=b6642e04f2e14619a454b3e9eed5a62a, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:03.537381 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba7ad4b-cb1f-420a-9069-ba214c1a0cdf service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:03.543823 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb9b9e-7a30-419c-b86b-2e61f4b0e9a2 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:03.555788 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1090/2157] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:38:03.576843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1068/2158] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:38:03 2022] PUT /identity/v3/projects/528b6c5135bd4cc0bdf783383aafc468/users/b6642e04f2e14619a454b3e9eed5a62a/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:03.609610 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe46b05-d0cd-45d6-bd36-7a6b6bb4d41b None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:03.609909 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe46b05-d0cd-45d6-bd36-7a6b6bb4d41b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:03.610298 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623cb4a8-8ca0-4680-afe4-8e12ce9f19f6 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:03.610597 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe46b05-d0cd-45d6-bd36-7a6b6bb4d41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:03.610983 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623cb4a8-8ca0-4680-afe4-8e12ce9f19f6 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:03.611430 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623cb4a8-8ca0-4680-afe4-8e12ce9f19f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:03.668747 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-efe46b05-d0cd-45d6-bd36-7a6b6bb4d41b None None] MFA Rules not processed for user `5fd3f793b3f04519bdb213f237d7d7ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:03.670589 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-efe46b05-d0cd-45d6-bd36-7a6b6bb4d41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:03.673791 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-623cb4a8-8ca0-4680-afe4-8e12ce9f19f6 None None] MFA Rules not processed for user `b6642e04f2e14619a454b3e9eed5a62a`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:03.676491 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-623cb4a8-8ca0-4680-afe4-8e12ce9f19f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:03.730622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:03.764779 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1091/2159] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:03 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:03.769993 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623cb4a8-8ca0-4680-afe4-8e12ce9f19f6 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:03.794139 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1069/2160] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:03.820180 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe46b05-d0cd-45d6-bd36-7a6b6bb4d41b None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:03.821820 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-efe46b05-d0cd-45d6-bd36-7a6b6bb4d41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:03.897336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623cb4a8-8ca0-4680-afe4-8e12ce9f19f6 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:03.915928 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17efc0d3-2513-4114-a1fe-00f69055ddfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:03.916654 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efc0d3-2513-4114-a1fe-00f69055ddfa admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:03.917015 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efc0d3-2513-4114-a1fe-00f69055ddfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:03.917255 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efc0d3-2513-4114-a1fe-00f69055ddfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:03.917961 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17efc0d3-2513-4114-a1fe-00f69055ddfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:03.919825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17efc0d3-2513-4114-a1fe-00f69055ddfa admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:03.925021 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:03.945687 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe46b05-d0cd-45d6-bd36-7a6b6bb4d41b None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:03.950584 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58c039-3127-4766-9a53-7c02d14b1fc7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dcfe5244bd04c00b6dde1d6f795d4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:03.951395 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58c039-3127-4766-9a53-7c02d14b1fc7 service glance] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:03.951826 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58c039-3127-4766-9a53-7c02d14b1fc7 service glance] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:03.952406 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58c039-3127-4766-9a53-7c02d14b1fc7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:03.957959 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fb58c039-3127-4766-9a53-7c02d14b1fc7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:03.973055 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1092/2161] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:38:03 2022] POST /identity/v3/projects => generated 403 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:03.974052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17efc0d3-2513-4114-a1fe-00f69055ddfa admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:03.976614 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-17efc0d3-2513-4114-a1fe-00f69055ddfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:04.052447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb58c039-3127-4766-9a53-7c02d14b1fc7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:04.054441 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb58c039-3127-4766-9a53-7c02d14b1fc7 service glance] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:04.064838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17efc0d3-2513-4114-a1fe-00f69055ddfa admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:04.067363 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1070/2162] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:38:04.068219 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58c039-3127-4766-9a53-7c02d14b1fc7 service glance] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:04.069895 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6b356-9fec-4e9d-8fc9-34ffa92e4ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:04.070542 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6b356-9fec-4e9d-8fc9-34ffa92e4ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:04.070823 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6b356-9fec-4e9d-8fc9-34ffa92e4ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:04.071596 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6b356-9fec-4e9d-8fc9-34ffa92e4ef7 admin admin] PATH_INFO: `/v3/projects/8b7d88f039644b71b2552a3c3083a70c` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:04.073157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a6b356-9fec-4e9d-8fc9-34ffa92e4ef7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b7d88f039644b71b2552a3c3083a70c)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:04.075372 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a6b356-9fec-4e9d-8fc9-34ffa92e4ef7 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:04.085785 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58c039-3127-4766-9a53-7c02d14b1fc7 service glance] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:04.091290 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f47bb-d69c-44e4-b38f-7d9698f1982e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:04.091906 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f47bb-d69c-44e4-b38f-7d9698f1982e admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:04.092186 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f47bb-d69c-44e4-b38f-7d9698f1982e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:04.092461 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f47bb-d69c-44e4-b38f-7d9698f1982e admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:04.093126 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f47bb-d69c-44e4-b38f-7d9698f1982e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:04.097116 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f47bb-d69c-44e4-b38f-7d9698f1982e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:04.156508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-84a6b356-9fec-4e9d-8fc9-34ffa92e4ef7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b7d88f039644b71b2552a3c3083a70c', 'request_id': 'req-84a6b356-9fec-4e9d-8fc9-34ffa92e4ef7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:04.157389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1071/2163] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:38:04 2022] POST /identity/v3/users => generated 308 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:04.160738 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f47bb-d69c-44e4-b38f-7d9698f1982e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:04.167010 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-84a6b356-9fec-4e9d-8fc9-34ffa92e4ef7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b7d88f039644b71b2552a3c3083a70c', 'request_id': 'req-84a6b356-9fec-4e9d-8fc9-34ffa92e4ef7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:04.174801 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-84a6b356-9fec-4e9d-8fc9-34ffa92e4ef7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b7d88f039644b71b2552a3c3083a70c', 'request_id': 'req-84a6b356-9fec-4e9d-8fc9-34ffa92e4ef7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:04.184721 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f47bb-d69c-44e4-b38f-7d9698f1982e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:04.194528 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-84a6b356-9fec-4e9d-8fc9-34ffa92e4ef7 admin admin] The token cache is being invalidate because project 8b7d88f039644b71b2552a3c3083a70c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:38:04.195015 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-84a6b356-9fec-4e9d-8fc9-34ffa92e4ef7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:04.199129 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-84a6b356-9fec-4e9d-8fc9-34ffa92e4ef7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:04.200898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1093/2164] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:38:03 2022] DELETE /identity/v3/projects/8b7d88f039644b71b2552a3c3083a70c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:04.210691 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6b356-9fec-4e9d-8fc9-34ffa92e4ef7 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:04.213304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-84a6b356-9fec-4e9d-8fc9-34ffa92e4ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:04.289413 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6a612-59c1-4214-ac41-aabc6f668278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:04.290757 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6a612-59c1-4214-ac41-aabc6f668278 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:04.291138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6a612-59c1-4214-ac41-aabc6f668278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:04.291515 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6a612-59c1-4214-ac41-aabc6f668278 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:04.292770 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6a612-59c1-4214-ac41-aabc6f668278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:04.294258 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6a612-59c1-4214-ac41-aabc6f668278 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:04.305716 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1072/2165] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:38:04.310198 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc930c-ecb1-4a87-ad31-1be759b934a8 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:04.310482 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc930c-ecb1-4a87-ad31-1be759b934a8 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:04.310759 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc930c-ecb1-4a87-ad31-1be759b934a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:04.323369 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:04.336317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6b356-9fec-4e9d-8fc9-34ffa92e4ef7 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:04.346285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30def63c-6817-4323-835e-6fe78ce2d856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:04.347560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30def63c-6817-4323-835e-6fe78ce2d856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:04.347560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30def63c-6817-4323-835e-6fe78ce2d856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:04.347560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30def63c-6817-4323-835e-6fe78ce2d856 admin admin] PATH_INFO: `/v3/users/8e6c7ebb29bf4c46adebfb56db366d6f` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:04.348803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30def63c-6817-4323-835e-6fe78ce2d856 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e6c7ebb29bf4c46adebfb56db366d6f)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:04.350784 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30def63c-6817-4323-835e-6fe78ce2d856 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:04.373816 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:04.383926 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f0bc930c-ecb1-4a87-ad31-1be759b934a8 None None] MFA Rules not processed for user `7a59f547a20e48ee88e3e620ba1e87f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:04.385413 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f0bc930c-ecb1-4a87-ad31-1be759b934a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:04.426370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-30def63c-6817-4323-835e-6fe78ce2d856 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e6c7ebb29bf4c46adebfb56db366d6f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:04.428341 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-30def63c-6817-4323-835e-6fe78ce2d856 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e6c7ebb29bf4c46adebfb56db366d6f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:04.447767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-30def63c-6817-4323-835e-6fe78ce2d856 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e6c7ebb29bf4c46adebfb56db366d6f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:04.465045 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-30def63c-6817-4323-835e-6fe78ce2d856 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e6c7ebb29bf4c46adebfb56db366d6f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:04.465869 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-30def63c-6817-4323-835e-6fe78ce2d856 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e6c7ebb29bf4c46adebfb56db366d6f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:04.473984 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1073/2166] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Oct 20 06:38:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:04.474895 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc930c-ecb1-4a87-ad31-1be759b934a8 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:04.476582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f0bc930c-ecb1-4a87-ad31-1be759b934a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:04.488073 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1094/2167] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:38:04 2022] DELETE /identity/v3/users/8e6c7ebb29bf4c46adebfb56db366d6f => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:04.503524 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc930c-ecb1-4a87-ad31-1be759b934a8 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:04.551308 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30def63c-6817-4323-835e-6fe78ce2d856 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:04.562843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-30def63c-6817-4323-835e-6fe78ce2d856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:04.600972 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f082443-b7a4-4fac-bf60-50c74c88e67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:04.601711 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f082443-b7a4-4fac-bf60-50c74c88e67e admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:04.602002 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f082443-b7a4-4fac-bf60-50c74c88e67e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:04.603014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f082443-b7a4-4fac-bf60-50c74c88e67e admin admin] PATH_INFO: `/v3/projects/a5a788166ca44812af92f59121ac9786/users/40fab22534d24db78a4ef930456a63b3/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:04.620665 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f082443-b7a4-4fac-bf60-50c74c88e67e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a788166ca44812af92f59121ac9786, user_id=40fab22534d24db78a4ef930456a63b3, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:04.624160 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f082443-b7a4-4fac-bf60-50c74c88e67e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:04.667529 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1074/2168] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:38:04 2022] PUT /identity/v3/projects/a5a788166ca44812af92f59121ac9786/users/40fab22534d24db78a4ef930456a63b3/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:04.673854 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f082443-b7a4-4fac-bf60-50c74c88e67e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:04.754981 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30def63c-6817-4323-835e-6fe78ce2d856 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:04.771417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:04.788917 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f082443-b7a4-4fac-bf60-50c74c88e67e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:04.796263 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d435b02-bc0b-4ccf-be2b-1ca7e8c3aed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:04.797397 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d435b02-bc0b-4ccf-be2b-1ca7e8c3aed0 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:04.797811 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d435b02-bc0b-4ccf-be2b-1ca7e8c3aed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:04.798243 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d435b02-bc0b-4ccf-be2b-1ca7e8c3aed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:04.799572 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d435b02-bc0b-4ccf-be2b-1ca7e8c3aed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:04.801949 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d435b02-bc0b-4ccf-be2b-1ca7e8c3aed0 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:04.824623 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1075/2169] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:38:04.825890 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d435b02-bc0b-4ccf-be2b-1ca7e8c3aed0 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:04.836554 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32898b49-7111-4c8f-a2df-96ea1e98eee3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeeab506ab5c4d6386fc899086f0abf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:04.837291 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d435b02-bc0b-4ccf-be2b-1ca7e8c3aed0 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:04.837721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32898b49-7111-4c8f-a2df-96ea1e98eee3 service swift] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:04.838317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32898b49-7111-4c8f-a2df-96ea1e98eee3 service swift] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:04.838870 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32898b49-7111-4c8f-a2df-96ea1e98eee3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:04.843724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c7274-2145-49bf-a0d3-dc5148465201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:04.844409 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c7274-2145-49bf-a0d3-dc5148465201 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:04.844688 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c7274-2145-49bf-a0d3-dc5148465201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:04.845004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c7274-2145-49bf-a0d3-dc5148465201 admin admin] PATH_INFO: `/v3/projects/a5a788166ca44812af92f59121ac9786/users/40fab22534d24db78a4ef930456a63b3/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:04.847195 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c7274-2145-49bf-a0d3-dc5148465201 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a788166ca44812af92f59121ac9786, user_id=40fab22534d24db78a4ef930456a63b3, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:04.849941 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c7274-2145-49bf-a0d3-dc5148465201 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:04.861016 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:04.861586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1076/2170] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:38:04 2022] PUT /identity/v3/projects/a5a788166ca44812af92f59121ac9786/users/40fab22534d24db78a4ef930456a63b3/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:04.920529 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32898b49-7111-4c8f-a2df-96ea1e98eee3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:04.922199 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32898b49-7111-4c8f-a2df-96ea1e98eee3 service swift] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:04.932289 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5b757-348f-4080-99b7-56bdd2fe1221 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:04.932592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1095/2171] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Oct 20 06:38:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 06:38:04.933895 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5b757-348f-4080-99b7-56bdd2fe1221 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:04.936037 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5b757-348f-4080-99b7-56bdd2fe1221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:04.971576 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-28c5b757-348f-4080-99b7-56bdd2fe1221 None None] MFA Rules not processed for user `40fab22534d24db78a4ef930456a63b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:04.973034 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-28c5b757-348f-4080-99b7-56bdd2fe1221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:04.991879 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32898b49-7111-4c8f-a2df-96ea1e98eee3 service swift] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:04.993610 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-32898b49-7111-4c8f-a2df-96ea1e98eee3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:05.026678 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32898b49-7111-4c8f-a2df-96ea1e98eee3 service swift] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:05.114173 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1077/2172] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:04 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:38:05.128465 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c5b757-348f-4080-99b7-56bdd2fe1221 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:05.129935 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-28c5b757-348f-4080-99b7-56bdd2fe1221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:05.217547 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ef912-45f9-463f-a570-a19c66c8f15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:05.218333 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ef912-45f9-463f-a570-a19c66c8f15b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:05.218709 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ef912-45f9-463f-a570-a19c66c8f15b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:05.219341 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ef912-45f9-463f-a570-a19c66c8f15b admin admin] PATH_INFO: `/v3/projects/e864b27e497c479ea4080a650a2fde9b` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:05.220535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ef912-45f9-463f-a570-a19c66c8f15b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e864b27e497c479ea4080a650a2fde9b)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:05.222984 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ef912-45f9-463f-a570-a19c66c8f15b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:05.278362 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c5b757-348f-4080-99b7-56bdd2fe1221 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:05.286202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6b07c-0cde-4d3a-88dc-c00d29a851eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:05.286870 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6b07c-0cde-4d3a-88dc-c00d29a851eb service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:05.287143 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6b07c-0cde-4d3a-88dc-c00d29a851eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:05.287458 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6b07c-0cde-4d3a-88dc-c00d29a851eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:05.330598 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d6b07c-0cde-4d3a-88dc-c00d29a851eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:05.331259 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-069ef912-45f9-463f-a570-a19c66c8f15b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e864b27e497c479ea4080a650a2fde9b', 'request_id': 'req-069ef912-45f9-463f-a570-a19c66c8f15b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:05.332098 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d6b07c-0cde-4d3a-88dc-c00d29a851eb service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:05.338209 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-069ef912-45f9-463f-a570-a19c66c8f15b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e864b27e497c479ea4080a650a2fde9b', 'request_id': 'req-069ef912-45f9-463f-a570-a19c66c8f15b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:05.351823 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1078/2173] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:05 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:38:05.352925 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6b07c-0cde-4d3a-88dc-c00d29a851eb service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:05.354288 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a5d6b07c-0cde-4d3a-88dc-c00d29a851eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:05.373447 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-069ef912-45f9-463f-a570-a19c66c8f15b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e864b27e497c479ea4080a650a2fde9b', 'request_id': 'req-069ef912-45f9-463f-a570-a19c66c8f15b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:05.441312 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-069ef912-45f9-463f-a570-a19c66c8f15b admin admin] The token cache is being invalidate because project e864b27e497c479ea4080a650a2fde9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:38:05.441697 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-069ef912-45f9-463f-a570-a19c66c8f15b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:05.444179 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-069ef912-45f9-463f-a570-a19c66c8f15b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:05.447509 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1096/2174] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:38:04 2022] DELETE /identity/v3/projects/e864b27e497c479ea4080a650a2fde9b => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:38:05.456548 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3fb811-8770-483c-a801-ddfab628f9fb None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:05.457385 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3fb811-8770-483c-a801-ddfab628f9fb None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:05.457668 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3fb811-8770-483c-a801-ddfab628f9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:05.514259 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-fc3fb811-8770-483c-a801-ddfab628f9fb None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:05.515486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-fc3fb811-8770-483c-a801-ddfab628f9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:05.526233 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6b07c-0cde-4d3a-88dc-c00d29a851eb service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:05.557892 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:05.599047 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:05.636863 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e669656-5941-46c6-80f7-efb07d2a3f94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:05.637493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e669656-5941-46c6-80f7-efb07d2a3f94 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:05.637772 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e669656-5941-46c6-80f7-efb07d2a3f94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:05.638047 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e669656-5941-46c6-80f7-efb07d2a3f94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:05.640159 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2e669656-5941-46c6-80f7-efb07d2a3f94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:05.640378 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1097/2175] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:05.642285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3fb811-8770-483c-a801-ddfab628f9fb None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:05.644033 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-fc3fb811-8770-483c-a801-ddfab628f9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:05.654909 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e669656-5941-46c6-80f7-efb07d2a3f94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:05.656537 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e669656-5941-46c6-80f7-efb07d2a3f94 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:05.667835 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1079/2176] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:38:05.669364 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191302b-749b-47c1-9d42-5816afb88534 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:05.669658 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191302b-749b-47c1-9d42-5816afb88534 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:05.669935 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191302b-749b-47c1-9d42-5816afb88534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:05.671157 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3fb811-8770-483c-a801-ddfab628f9fb None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:05.678945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a20cf-7e74-4732-875b-4cc0af3f9629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:05.679862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a20cf-7e74-4732-875b-4cc0af3f9629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:05.680232 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a20cf-7e74-4732-875b-4cc0af3f9629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:05.680744 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a20cf-7e74-4732-875b-4cc0af3f9629 admin admin] PATH_INFO: `/v3/users/b6642e04f2e14619a454b3e9eed5a62a` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:05.682042 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a20cf-7e74-4732-875b-4cc0af3f9629 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6642e04f2e14619a454b3e9eed5a62a)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:05.684140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a20cf-7e74-4732-875b-4cc0af3f9629 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:05.693062 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-5191302b-749b-47c1-9d42-5816afb88534 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:05.695003 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5191302b-749b-47c1-9d42-5816afb88534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:05.719096 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1080/2177] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:05.724754 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191302b-749b-47c1-9d42-5816afb88534 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:05.744727 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191302b-749b-47c1-9d42-5816afb88534 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:05.754070 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c0df0-e004-48c9-bc16-6a61608d6a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:05.755989 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c0df0-e004-48c9-bc16-6a61608d6a5f admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:05.756466 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c0df0-e004-48c9-bc16-6a61608d6a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:05.756952 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c0df0-e004-48c9-bc16-6a61608d6a5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:05.757387 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-095a20cf-7e74-4732-875b-4cc0af3f9629 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6642e04f2e14619a454b3e9eed5a62a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:05.757622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c0df0-e004-48c9-bc16-6a61608d6a5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:38:05.757928 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c0df0-e004-48c9-bc16-6a61608d6a5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:05.758372 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-095a20cf-7e74-4732-875b-4cc0af3f9629 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6642e04f2e14619a454b3e9eed5a62a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:05.759900 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c0df0-e004-48c9-bc16-6a61608d6a5f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:05.777707 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-095a20cf-7e74-4732-875b-4cc0af3f9629 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6642e04f2e14619a454b3e9eed5a62a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:05.785876 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1081/2178] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 06:38:05.795585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c0df0-e004-48c9-bc16-6a61608d6a5f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:05.798291 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b79c0df0-e004-48c9-bc16-6a61608d6a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:05.799353 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-095a20cf-7e74-4732-875b-4cc0af3f9629 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6642e04f2e14619a454b3e9eed5a62a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:05.800200 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-095a20cf-7e74-4732-875b-4cc0af3f9629 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6642e04f2e14619a454b3e9eed5a62a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:05.820547 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1098/2179] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:38:05 2022] DELETE /identity/v3/users/b6642e04f2e14619a454b3e9eed5a62a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:05.825632 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74353aac-abf8-448e-ad7e-7a9fd84c2c68 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:05.826037 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74353aac-abf8-448e-ad7e-7a9fd84c2c68 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:05.826404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74353aac-abf8-448e-ad7e-7a9fd84c2c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:05.863943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c0df0-e004-48c9-bc16-6a61608d6a5f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:05.884238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-74353aac-abf8-448e-ad7e-7a9fd84c2c68 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:05.890012 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-74353aac-abf8-448e-ad7e-7a9fd84c2c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:06.009588 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f323f3-d5a6-40e3-9f32-0a1fa9b1fc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:06.010402 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f323f3-d5a6-40e3-9f32-0a1fa9b1fc4c admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:06.010770 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f323f3-d5a6-40e3-9f32-0a1fa9b1fc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:06.011216 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f323f3-d5a6-40e3-9f32-0a1fa9b1fc4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:06.011955 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f323f3-d5a6-40e3-9f32-0a1fa9b1fc4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:06.014530 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f323f3-d5a6-40e3-9f32-0a1fa9b1fc4c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:06.075709 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1082/2180] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:38:05 2022] POST /identity/v3/projects => generated 383 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:06.088959 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f323f3-d5a6-40e3-9f32-0a1fa9b1fc4c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:06.089154 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1099/2181] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:38:06.095812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74353aac-abf8-448e-ad7e-7a9fd84c2c68 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:06.098056 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-74353aac-abf8-448e-ad7e-7a9fd84c2c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:06.211142 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74353aac-abf8-448e-ad7e-7a9fd84c2c68 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:06.232721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:06.240132 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f323f3-d5a6-40e3-9f32-0a1fa9b1fc4c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:06.251863 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe255514-c698-4ea1-8f71-4fc3f6a3c1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:06.252504 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe255514-c698-4ea1-8f71-4fc3f6a3c1de admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:06.252814 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe255514-c698-4ea1-8f71-4fc3f6a3c1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:06.253096 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe255514-c698-4ea1-8f71-4fc3f6a3c1de admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:06.253746 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe255514-c698-4ea1-8f71-4fc3f6a3c1de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:06.257445 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe255514-c698-4ea1-8f71-4fc3f6a3c1de admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:06.300428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471f355-1416-47d8-836b-8248cf1a3b90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:06.300428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471f355-1416-47d8-836b-8248cf1a3b90 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:06.300428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471f355-1416-47d8-836b-8248cf1a3b90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:06.300428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471f355-1416-47d8-836b-8248cf1a3b90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:06.306887 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1083/2182] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:38:06 2022] POST /identity/v3/users => generated 292 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:06.315458 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe255514-c698-4ea1-8f71-4fc3f6a3c1de admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:06.324740 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471f355-1416-47d8-836b-8248cf1a3b90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:06.326206 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471f355-1416-47d8-836b-8248cf1a3b90 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:06.331100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe255514-c698-4ea1-8f71-4fc3f6a3c1de admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:06.339197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e9ee6-13a1-4032-ac46-366088aef84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:06.340567 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e9ee6-13a1-4032-ac46-366088aef84d admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:06.340982 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e9ee6-13a1-4032-ac46-366088aef84d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:06.341380 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e9ee6-13a1-4032-ac46-366088aef84d admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:06.343108 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e9ee6-13a1-4032-ac46-366088aef84d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:06.345669 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e9ee6-13a1-4032-ac46-366088aef84d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:06.348761 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1100/2183] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:38:06.358474 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1084/2184] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:38:06.362564 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471f355-1416-47d8-836b-8248cf1a3b90 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:06.373759 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471f355-1416-47d8-836b-8248cf1a3b90 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:06.380845 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26f7ac-fbe6-4e2b-ae23-cc60663ff0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:06.381517 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26f7ac-fbe6-4e2b-ae23-cc60663ff0c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:06.381800 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26f7ac-fbe6-4e2b-ae23-cc60663ff0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:06.382075 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26f7ac-fbe6-4e2b-ae23-cc60663ff0c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:06.382773 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26f7ac-fbe6-4e2b-ae23-cc60663ff0c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:06.386116 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26f7ac-fbe6-4e2b-ae23-cc60663ff0c0 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:06.405490 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1101/2185] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:38:06.409633 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e9ee6-13a1-4032-ac46-366088aef84d admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:06.424256 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e9ee6-13a1-4032-ac46-366088aef84d admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:06.440324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c32226-032c-4259-8c04-9a99e6aeb038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:06.442902 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c32226-032c-4259-8c04-9a99e6aeb038 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:06.444876 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c32226-032c-4259-8c04-9a99e6aeb038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:06.445193 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c32226-032c-4259-8c04-9a99e6aeb038 admin admin] PATH_INFO: `/v3/projects/7ca261b5add24645a546a7887a67f44b/users/b15e9df78ead43ca97787a2f624fad8b/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:06.450431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:06.471912 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c32226-032c-4259-8c04-9a99e6aeb038 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca261b5add24645a546a7887a67f44b, user_id=b15e9df78ead43ca97787a2f624fad8b, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:06.474975 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c32226-032c-4259-8c04-9a99e6aeb038 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:06.501413 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1085/2186] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:38:06 2022] PUT /identity/v3/projects/7ca261b5add24645a546a7887a67f44b/users/b15e9df78ead43ca97787a2f624fad8b/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:06.516248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c629ee-43d4-4916-ae6a-8810099055ad None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:06.516534 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c629ee-43d4-4916-ae6a-8810099055ad None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:06.516843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c629ee-43d4-4916-ae6a-8810099055ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:06.596757 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-81c629ee-43d4-4916-ae6a-8810099055ad None None] MFA Rules not processed for user `b15e9df78ead43ca97787a2f624fad8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:06.598792 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-81c629ee-43d4-4916-ae6a-8810099055ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:06.693296 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1102/2187] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:06.698619 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c32226-032c-4259-8c04-9a99e6aeb038 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:06.870882 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c32226-032c-4259-8c04-9a99e6aeb038 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:06.883167 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51475d2b-3edb-4368-b31e-efb69f15937c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:06.883911 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51475d2b-3edb-4368-b31e-efb69f15937c admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:06.884299 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51475d2b-3edb-4368-b31e-efb69f15937c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:06.886100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51475d2b-3edb-4368-b31e-efb69f15937c admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:06.886801 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51475d2b-3edb-4368-b31e-efb69f15937c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:06.889636 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51475d2b-3edb-4368-b31e-efb69f15937c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:06.959268 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1086/2188] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:38:06 2022] POST /identity/v3/projects => generated 377 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:06.960441 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c629ee-43d4-4916-ae6a-8810099055ad None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:06.978212 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c629ee-43d4-4916-ae6a-8810099055ad None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:06.991480 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51475d2b-3edb-4368-b31e-efb69f15937c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:06.993865 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-51475d2b-3edb-4368-b31e-efb69f15937c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:07.100958 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb6a7f-475b-475d-9055-09c22b9c0d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:07.101673 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb6a7f-475b-475d-9055-09c22b9c0d1a admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:07.101936 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb6a7f-475b-475d-9055-09c22b9c0d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:07.103331 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb6a7f-475b-475d-9055-09c22b9c0d1a admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:07.103703 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb6a7f-475b-475d-9055-09c22b9c0d1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:07.107387 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb6a7f-475b-475d-9055-09c22b9c0d1a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:07.144325 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1103/2189] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:38:06 2022] POST /identity/v3/users => generated 295 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:07.152285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb6a7f-475b-475d-9055-09c22b9c0d1a admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:07.174263 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51475d2b-3edb-4368-b31e-efb69f15937c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:07.179587 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb6a7f-475b-475d-9055-09c22b9c0d1a admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:07.186453 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352f4a3-c0b3-4430-a5ed-6f6974a0b50e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfbaaf32475a4c82a34948602d8f4e35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:07.187125 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352f4a3-c0b3-4430-a5ed-6f6974a0b50e service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:07.187412 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352f4a3-c0b3-4430-a5ed-6f6974a0b50e service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:07.187862 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352f4a3-c0b3-4430-a5ed-6f6974a0b50e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:07.191988 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad4a55-271d-437c-9277-c7287aa73f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:07.192415 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad4a55-271d-437c-9277-c7287aa73f06 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:07.192725 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad4a55-271d-437c-9277-c7287aa73f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:07.193001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad4a55-271d-437c-9277-c7287aa73f06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:07.193662 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad4a55-271d-437c-9277-c7287aa73f06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:07.195222 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c352f4a3-c0b3-4430-a5ed-6f6974a0b50e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:07.199702 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad4a55-271d-437c-9277-c7287aa73f06 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:07.227171 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1104/2190] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:38:07.232847 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c352f4a3-c0b3-4430-a5ed-6f6974a0b50e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:07.234515 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c352f4a3-c0b3-4430-a5ed-6f6974a0b50e service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:07.238754 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad4a55-271d-437c-9277-c7287aa73f06 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:07.241701 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-83ad4a55-271d-437c-9277-c7287aa73f06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:07.268822 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1087/2191] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:38:07.269535 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352f4a3-c0b3-4430-a5ed-6f6974a0b50e service placement] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:07.279076 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad4a55-271d-437c-9277-c7287aa73f06 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:07.294610 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f7d56-45cb-4fe4-be40-35013f54d037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:07.295272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f7d56-45cb-4fe4-be40-35013f54d037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:07.295549 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f7d56-45cb-4fe4-be40-35013f54d037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:07.295824 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f7d56-45cb-4fe4-be40-35013f54d037 admin admin] PATH_INFO: `/v3/users/7935a2a80f054f699983e5e2f3e64a28` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:07.300669 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f7d56-45cb-4fe4-be40-35013f54d037 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7935a2a80f054f699983e5e2f3e64a28)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:07.300669 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f7d56-45cb-4fe4-be40-35013f54d037 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:07.302267 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352f4a3-c0b3-4430-a5ed-6f6974a0b50e service placement] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:07.312219 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac83d3-9d0c-4a24-991e-dd1daf2018cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:07.313009 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac83d3-9d0c-4a24-991e-dd1daf2018cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:07.313448 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac83d3-9d0c-4a24-991e-dd1daf2018cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:07.313739 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac83d3-9d0c-4a24-991e-dd1daf2018cf admin admin] PATH_INFO: `/v3/projects/94e1183c6b1844188399836f00d61012/users/a3b6dbc3f94d4efa8502cf1b38c9b721/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:07.334907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:07.355055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac83d3-9d0c-4a24-991e-dd1daf2018cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=94e1183c6b1844188399836f00d61012, user_id=a3b6dbc3f94d4efa8502cf1b38c9b721, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:07.357748 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac83d3-9d0c-4a24-991e-dd1daf2018cf admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:07.404395 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1088/2192] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:38:07 2022] PUT /identity/v3/projects/94e1183c6b1844188399836f00d61012/users/a3b6dbc3f94d4efa8502cf1b38c9b721/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:07.409631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac83d3-9d0c-4a24-991e-dd1daf2018cf admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:07.425918 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-144f7d56-45cb-4fe4-be40-35013f54d037 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7935a2a80f054f699983e5e2f3e64a28'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:07.427660 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-144f7d56-45cb-4fe4-be40-35013f54d037 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7935a2a80f054f699983e5e2f3e64a28'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:07.448332 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-144f7d56-45cb-4fe4-be40-35013f54d037 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7935a2a80f054f699983e5e2f3e64a28'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:07.478782 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-144f7d56-45cb-4fe4-be40-35013f54d037 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7935a2a80f054f699983e5e2f3e64a28'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:07.479647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-144f7d56-45cb-4fe4-be40-35013f54d037 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7935a2a80f054f699983e5e2f3e64a28'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:07.506337 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:07.514341 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1105/2193] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:38:07 2022] DELETE /identity/v3/users/7935a2a80f054f699983e5e2f3e64a28 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:07.587650 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac83d3-9d0c-4a24-991e-dd1daf2018cf admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:07.695244 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e135d-2df5-475e-8b09-1645edbf0ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:07.695879 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e135d-2df5-475e-8b09-1645edbf0ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:07.696939 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e135d-2df5-475e-8b09-1645edbf0ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:07.698111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e135d-2df5-475e-8b09-1645edbf0ec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:07.698930 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e135d-2df5-475e-8b09-1645edbf0ec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:07.700565 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e135d-2df5-475e-8b09-1645edbf0ec0 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:07.738017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f7d56-45cb-4fe4-be40-35013f54d037 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:07.738687 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1089/2194] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:38:07.740648 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-144f7d56-45cb-4fe4-be40-35013f54d037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:07.764025 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f7d56-45cb-4fe4-be40-35013f54d037 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:07.778913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25e595-d799-45d5-a41f-7c168eedc8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:07.779750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25e595-d799-45d5-a41f-7c168eedc8ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:07.780038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25e595-d799-45d5-a41f-7c168eedc8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:07.780315 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25e595-d799-45d5-a41f-7c168eedc8ac admin admin] PATH_INFO: `/v3/projects/94e1183c6b1844188399836f00d61012/users/a3b6dbc3f94d4efa8502cf1b38c9b721/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:07.782524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25e595-d799-45d5-a41f-7c168eedc8ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=94e1183c6b1844188399836f00d61012, user_id=a3b6dbc3f94d4efa8502cf1b38c9b721, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:07.785120 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25e595-d799-45d5-a41f-7c168eedc8ac admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:07.821769 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1106/2195] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:38:07 2022] PUT /identity/v3/projects/94e1183c6b1844188399836f00d61012/users/a3b6dbc3f94d4efa8502cf1b38c9b721/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:07.823420 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16fa42-3ec1-4727-82b3-e19fe21c768c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:07.823713 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16fa42-3ec1-4727-82b3-e19fe21c768c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:07.823994 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16fa42-3ec1-4727-82b3-e19fe21c768c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:07.873160 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3c16fa42-3ec1-4727-82b3-e19fe21c768c None None] MFA Rules not processed for user `a3b6dbc3f94d4efa8502cf1b38c9b721`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:07.875057 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3c16fa42-3ec1-4727-82b3-e19fe21c768c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:08.031905 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25e595-d799-45d5-a41f-7c168eedc8ac admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:08.034937 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1a25e595-d799-45d5-a41f-7c168eedc8ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:08.106506 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1090/2196] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:08.108813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16fa42-3ec1-4727-82b3-e19fe21c768c None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:08.125834 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3c16fa42-3ec1-4727-82b3-e19fe21c768c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:08.258130 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25e595-d799-45d5-a41f-7c168eedc8ac admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:08.268215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d17862-357b-4b26-8ea9-0578c2911852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:08.269232 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d17862-357b-4b26-8ea9-0578c2911852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:08.269803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d17862-357b-4b26-8ea9-0578c2911852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:08.270081 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d17862-357b-4b26-8ea9-0578c2911852 admin admin] PATH_INFO: `/v3/projects/3de3d03dde2d4ef6a8b925a0a5391d0b` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:08.271315 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d17862-357b-4b26-8ea9-0578c2911852 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3de3d03dde2d4ef6a8b925a0a5391d0b)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:08.275971 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d17862-357b-4b26-8ea9-0578c2911852 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:08.281002 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16fa42-3ec1-4727-82b3-e19fe21c768c None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:08.286427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9825d-a5cb-43b1-bee0-95bc5644d87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:08.287530 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9825d-a5cb-43b1-bee0-95bc5644d87c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:08.288060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9825d-a5cb-43b1-bee0-95bc5644d87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:08.288528 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9825d-a5cb-43b1-bee0-95bc5644d87c admin admin] PATH_INFO: `/v3/projects/528b6c5135bd4cc0bdf783383aafc468` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:08.290329 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa9825d-a5cb-43b1-bee0-95bc5644d87c admin admin] RBAC: Authorizing `identity:delete_project(project_id=528b6c5135bd4cc0bdf783383aafc468)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:08.292749 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa9825d-a5cb-43b1-bee0-95bc5644d87c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:08.309450 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:08.369429 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-43d17862-357b-4b26-8ea9-0578c2911852 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3de3d03dde2d4ef6a8b925a0a5391d0b', 'request_id': 'req-43d17862-357b-4b26-8ea9-0578c2911852'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:08.376589 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-43d17862-357b-4b26-8ea9-0578c2911852 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3de3d03dde2d4ef6a8b925a0a5391d0b', 'request_id': 'req-43d17862-357b-4b26-8ea9-0578c2911852'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:08.392927 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-43d17862-357b-4b26-8ea9-0578c2911852 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3de3d03dde2d4ef6a8b925a0a5391d0b', 'request_id': 'req-43d17862-357b-4b26-8ea9-0578c2911852'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:08.397184 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3fa9825d-a5cb-43b1-bee0-95bc5644d87c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '528b6c5135bd4cc0bdf783383aafc468', 'request_id': 'req-3fa9825d-a5cb-43b1-bee0-95bc5644d87c'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:08.405281 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3fa9825d-a5cb-43b1-bee0-95bc5644d87c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '528b6c5135bd4cc0bdf783383aafc468', 'request_id': 'req-3fa9825d-a5cb-43b1-bee0-95bc5644d87c'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:08.412498 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-43d17862-357b-4b26-8ea9-0578c2911852 admin admin] The token cache is being invalidate because project 3de3d03dde2d4ef6a8b925a0a5391d0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:38:08.412852 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3fa9825d-a5cb-43b1-bee0-95bc5644d87c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '528b6c5135bd4cc0bdf783383aafc468', 'request_id': 'req-3fa9825d-a5cb-43b1-bee0-95bc5644d87c'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:08.413186 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-43d17862-357b-4b26-8ea9-0578c2911852 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:08.414142 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-43d17862-357b-4b26-8ea9-0578c2911852 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:08.418429 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1107/2197] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:38:08 2022] DELETE /identity/v3/projects/3de3d03dde2d4ef6a8b925a0a5391d0b => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:38:08.420885 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d17862-357b-4b26-8ea9-0578c2911852 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:08.424237 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-43d17862-357b-4b26-8ea9-0578c2911852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:08.445418 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:08.465106 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3fa9825d-a5cb-43b1-bee0-95bc5644d87c admin admin] The token cache is being invalidate because project 528b6c5135bd4cc0bdf783383aafc468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:38:08.471581 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3fa9825d-a5cb-43b1-bee0-95bc5644d87c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:08.472552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3fa9825d-a5cb-43b1-bee0-95bc5644d87c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:08.477757 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1091/2198] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:38:08 2022] DELETE /identity/v3/projects/528b6c5135bd4cc0bdf783383aafc468 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:08.492920 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9825d-a5cb-43b1-bee0-95bc5644d87c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:08.495785 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3fa9825d-a5cb-43b1-bee0-95bc5644d87c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:08.518767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d17862-357b-4b26-8ea9-0578c2911852 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:08.634183 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9041a48-7f4b-452c-93f3-1c0be6a0cf56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:08.635085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9041a48-7f4b-452c-93f3-1c0be6a0cf56 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:08.635951 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9041a48-7f4b-452c-93f3-1c0be6a0cf56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:08.636322 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9041a48-7f4b-452c-93f3-1c0be6a0cf56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:08.638599 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d9041a48-7f4b-452c-93f3-1c0be6a0cf56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:08.747008 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9041a48-7f4b-452c-93f3-1c0be6a0cf56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:08.749488 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9041a48-7f4b-452c-93f3-1c0be6a0cf56 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:08.765513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1108/2199] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:38:08.789131 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9825d-a5cb-43b1-bee0-95bc5644d87c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:08.810473 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eba5e0-3a5d-432c-a474-eaf9b0a8b92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:08.811141 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eba5e0-3a5d-432c-a474-eaf9b0a8b92b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:08.811439 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eba5e0-3a5d-432c-a474-eaf9b0a8b92b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:08.811715 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eba5e0-3a5d-432c-a474-eaf9b0a8b92b admin admin] PATH_INFO: `/v3/users/40fab22534d24db78a4ef930456a63b3` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:08.815970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eba5e0-3a5d-432c-a474-eaf9b0a8b92b admin admin] RBAC: Authorizing `identity:delete_user(user_id=40fab22534d24db78a4ef930456a63b3)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:08.818087 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eba5e0-3a5d-432c-a474-eaf9b0a8b92b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:08.934147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e7eba5e0-3a5d-432c-a474-eaf9b0a8b92b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40fab22534d24db78a4ef930456a63b3'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:08.934859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e7eba5e0-3a5d-432c-a474-eaf9b0a8b92b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40fab22534d24db78a4ef930456a63b3'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:08.946951 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e7eba5e0-3a5d-432c-a474-eaf9b0a8b92b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40fab22534d24db78a4ef930456a63b3'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:08.968844 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9041a48-7f4b-452c-93f3-1c0be6a0cf56 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:08.970491 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d9041a48-7f4b-452c-93f3-1c0be6a0cf56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:08.981881 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e7eba5e0-3a5d-432c-a474-eaf9b0a8b92b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40fab22534d24db78a4ef930456a63b3'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:08.984823 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9041a48-7f4b-452c-93f3-1c0be6a0cf56 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:08.995467 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf62186-031c-4fe1-bd95-20243d9b5ca2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:08.997326 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e7eba5e0-3a5d-432c-a474-eaf9b0a8b92b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40fab22534d24db78a4ef930456a63b3'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:08.997921 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf62186-031c-4fe1-bd95-20243d9b5ca2 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:08.998654 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf62186-031c-4fe1-bd95-20243d9b5ca2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:09.000299 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf62186-031c-4fe1-bd95-20243d9b5ca2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:09.004926 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-faf62186-031c-4fe1-bd95-20243d9b5ca2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:09.010133 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1092/2200] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:38:08 2022] DELETE /identity/v3/users/40fab22534d24db78a4ef930456a63b3 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:09.186348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf62186-031c-4fe1-bd95-20243d9b5ca2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:09.188572 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf62186-031c-4fe1-bd95-20243d9b5ca2 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:09.208612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1109/2201] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:38:09.463578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eba5e0-3a5d-432c-a474-eaf9b0a8b92b admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:09.465677 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e7eba5e0-3a5d-432c-a474-eaf9b0a8b92b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:09.497105 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:09.630845 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eba5e0-3a5d-432c-a474-eaf9b0a8b92b admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:09.663067 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b4bac-f798-4025-90de-d105df90b5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:09.663791 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b4bac-f798-4025-90de-d105df90b5da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:09.664173 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b4bac-f798-4025-90de-d105df90b5da admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:09.665438 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b4bac-f798-4025-90de-d105df90b5da admin admin] PATH_INFO: `/v3/projects/a5a788166ca44812af92f59121ac9786` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:09.667171 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b4bac-f798-4025-90de-d105df90b5da admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5a788166ca44812af92f59121ac9786)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:09.669891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b4bac-f798-4025-90de-d105df90b5da admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:09.808724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-bc6b4bac-f798-4025-90de-d105df90b5da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5a788166ca44812af92f59121ac9786', 'request_id': 'req-bc6b4bac-f798-4025-90de-d105df90b5da'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:09.820514 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-bc6b4bac-f798-4025-90de-d105df90b5da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5a788166ca44812af92f59121ac9786', 'request_id': 'req-bc6b4bac-f798-4025-90de-d105df90b5da'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:09.833488 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-bc6b4bac-f798-4025-90de-d105df90b5da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5a788166ca44812af92f59121ac9786', 'request_id': 'req-bc6b4bac-f798-4025-90de-d105df90b5da'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:09.861940 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-bc6b4bac-f798-4025-90de-d105df90b5da admin admin] The token cache is being invalidate because project a5a788166ca44812af92f59121ac9786 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:38:09.862107 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-bc6b4bac-f798-4025-90de-d105df90b5da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:09.862784 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-bc6b4bac-f798-4025-90de-d105df90b5da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:09.869548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1093/2202] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:38:09 2022] DELETE /identity/v3/projects/a5a788166ca44812af92f59121ac9786 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:09.875010 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2615c9-f2e0-4e53-a556-43bbbe9433a9 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:09.875478 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2615c9-f2e0-4e53-a556-43bbbe9433a9 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:09.875632 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2615c9-f2e0-4e53-a556-43bbbe9433a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:09.891531 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b4bac-f798-4025-90de-d105df90b5da admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:09.900565 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-bc6b4bac-f798-4025-90de-d105df90b5da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:09.911128 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-0d2615c9-f2e0-4e53-a556-43bbbe9433a9 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:09.912347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-0d2615c9-f2e0-4e53-a556-43bbbe9433a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:09.923096 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:10.054154 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1110/2203] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:10.062107 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4478852-5827-452e-b066-564ecba7c4b1 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:10.062390 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4478852-5827-452e-b066-564ecba7c4b1 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:10.070424 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4478852-5827-452e-b066-564ecba7c4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:10.109977 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a4478852-5827-452e-b066-564ecba7c4b1 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:10.111196 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a4478852-5827-452e-b066-564ecba7c4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:10.124727 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1111/2204] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:38:10.128128 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4478852-5827-452e-b066-564ecba7c4b1 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:10.134039 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b4bac-f798-4025-90de-d105df90b5da admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:10.139056 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dabfa32-af32-4d3d-a3c9-d58757a88d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:10.140007 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabfa32-af32-4d3d-a3c9-d58757a88d22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:10.140460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabfa32-af32-4d3d-a3c9-d58757a88d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:10.140960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabfa32-af32-4d3d-a3c9-d58757a88d22 admin admin] PATH_INFO: `/v3/users/b15e9df78ead43ca97787a2f624fad8b` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:10.143752 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dabfa32-af32-4d3d-a3c9-d58757a88d22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b15e9df78ead43ca97787a2f624fad8b)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:10.145384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4478852-5827-452e-b066-564ecba7c4b1 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:10.149441 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dabfa32-af32-4d3d-a3c9-d58757a88d22 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:10.150283 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb04afc8-1d60-4c6b-892d-b03c0d768de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:10.150901 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04afc8-1d60-4c6b-892d-b03c0d768de4 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:10.151174 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04afc8-1d60-4c6b-892d-b03c0d768de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:10.151442 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04afc8-1d60-4c6b-892d-b03c0d768de4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:10.152083 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb04afc8-1d60-4c6b-892d-b03c0d768de4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:38:10.152381 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb04afc8-1d60-4c6b-892d-b03c0d768de4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:10.154140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb04afc8-1d60-4c6b-892d-b03c0d768de4 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:10.170021 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1112/2205] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:38:10.174718 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb04afc8-1d60-4c6b-892d-b03c0d768de4 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:10.199281 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb04afc8-1d60-4c6b-892d-b03c0d768de4 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:10.206161 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b57f62-2b00-46a0-b741-7303bc7fc28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:10.207648 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b57f62-2b00-46a0-b741-7303bc7fc28f admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:10.208050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b57f62-2b00-46a0-b741-7303bc7fc28f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:10.208727 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b57f62-2b00-46a0-b741-7303bc7fc28f admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:10.210320 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b57f62-2b00-46a0-b741-7303bc7fc28f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:10.213322 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b57f62-2b00-46a0-b741-7303bc7fc28f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:10.252563 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:10.258970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5dabfa32-af32-4d3d-a3c9-d58757a88d22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b15e9df78ead43ca97787a2f624fad8b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:10.260565 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5dabfa32-af32-4d3d-a3c9-d58757a88d22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b15e9df78ead43ca97787a2f624fad8b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:10.265993 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1113/2206] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:38:10 2022] POST /identity/v3/projects => generated 403 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:10.266608 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b57f62-2b00-46a0-b741-7303bc7fc28f admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:10.276198 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5dabfa32-af32-4d3d-a3c9-d58757a88d22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b15e9df78ead43ca97787a2f624fad8b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:10.296730 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5dabfa32-af32-4d3d-a3c9-d58757a88d22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b15e9df78ead43ca97787a2f624fad8b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:10.313848 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5dabfa32-af32-4d3d-a3c9-d58757a88d22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b15e9df78ead43ca97787a2f624fad8b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:10.325368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1094/2207] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:38:09 2022] DELETE /identity/v3/users/b15e9df78ead43ca97787a2f624fad8b => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:10.340228 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8ba6f-3bc1-4992-9313-b4ecbc4ef546 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:10.343301 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8ba6f-3bc1-4992-9313-b4ecbc4ef546 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:10.346018 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8ba6f-3bc1-4992-9313-b4ecbc4ef546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:10.397681 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3cc8ba6f-3bc1-4992-9313-b4ecbc4ef546 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:10.399132 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3cc8ba6f-3bc1-4992-9313-b4ecbc4ef546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:10.400127 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b57f62-2b00-46a0-b741-7303bc7fc28f admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:10.496262 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1095/2208] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:10.517921 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8ba6f-3bc1-4992-9313-b4ecbc4ef546 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:10.519812 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3cc8ba6f-3bc1-4992-9313-b4ecbc4ef546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:10.524900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa5525-dab8-4ebd-a02d-e6941933b040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:10.525613 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa5525-dab8-4ebd-a02d-e6941933b040 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:10.525985 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa5525-dab8-4ebd-a02d-e6941933b040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:10.526351 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa5525-dab8-4ebd-a02d-e6941933b040 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:10.527081 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa5525-dab8-4ebd-a02d-e6941933b040 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:10.529190 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa5525-dab8-4ebd-a02d-e6941933b040 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:10.574251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1114/2209] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:38:10 2022] POST /identity/v3/users => generated 302 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:10.575790 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa5525-dab8-4ebd-a02d-e6941933b040 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:10.578058 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e3aa5525-dab8-4ebd-a02d-e6941933b040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:10.595653 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa5525-dab8-4ebd-a02d-e6941933b040 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:10.611346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453be49-0245-4f1b-b26e-2cf49f850694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:10.612270 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453be49-0245-4f1b-b26e-2cf49f850694 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:10.612646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453be49-0245-4f1b-b26e-2cf49f850694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:10.613038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453be49-0245-4f1b-b26e-2cf49f850694 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:10.613787 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453be49-0245-4f1b-b26e-2cf49f850694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:10.615731 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453be49-0245-4f1b-b26e-2cf49f850694 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:10.620385 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:10.638803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1115/2210] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:38:10.641192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453be49-0245-4f1b-b26e-2cf49f850694 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:10.644554 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8ba6f-3bc1-4992-9313-b4ecbc4ef546 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:10.655104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5751b5-e036-4307-96f0-f112bdaf0d75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:10.655835 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5751b5-e036-4307-96f0-f112bdaf0d75 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:10.656364 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5751b5-e036-4307-96f0-f112bdaf0d75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:10.656748 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5751b5-e036-4307-96f0-f112bdaf0d75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:10.662030 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453be49-0245-4f1b-b26e-2cf49f850694 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:10.674860 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50e777-bca7-435a-896c-ef52fa5545ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:10.675712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50e777-bca7-435a-896c-ef52fa5545ad admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:10.676086 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50e777-bca7-435a-896c-ef52fa5545ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:10.676452 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50e777-bca7-435a-896c-ef52fa5545ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:10.677425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a50e777-bca7-435a-896c-ef52fa5545ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:10.678429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5751b5-e036-4307-96f0-f112bdaf0d75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:10.680769 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5751b5-e036-4307-96f0-f112bdaf0d75 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:10.682004 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a50e777-bca7-435a-896c-ef52fa5545ad admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:10.704516 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1096/2211] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:38:10.715700 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1116/2212] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:38:10.716916 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50e777-bca7-435a-896c-ef52fa5545ad admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:10.737268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50e777-bca7-435a-896c-ef52fa5545ad admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:10.751747 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98820e-59d8-4dbd-a2c5-cc2e77a64af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:10.752521 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98820e-59d8-4dbd-a2c5-cc2e77a64af3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:10.752926 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98820e-59d8-4dbd-a2c5-cc2e77a64af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:10.753377 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98820e-59d8-4dbd-a2c5-cc2e77a64af3 admin admin] PATH_INFO: `/v3/projects/41866630bfff4e31b10bbde20a42290b/users/757d987b71484f8aaa24062d6e509aa8/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:10.792235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98820e-59d8-4dbd-a2c5-cc2e77a64af3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41866630bfff4e31b10bbde20a42290b, user_id=757d987b71484f8aaa24062d6e509aa8, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:10.797607 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98820e-59d8-4dbd-a2c5-cc2e77a64af3 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:10.830118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1117/2213] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:38:10 2022] PUT /identity/v3/projects/41866630bfff4e31b10bbde20a42290b/users/757d987b71484f8aaa24062d6e509aa8/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:10.832252 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c010fa-9eac-4a36-8f31-146fa9cab6e5 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:10.832828 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c010fa-9eac-4a36-8f31-146fa9cab6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:10.833145 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c010fa-9eac-4a36-8f31-146fa9cab6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:10.880747 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f1c010fa-9eac-4a36-8f31-146fa9cab6e5 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:10.884519 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f1c010fa-9eac-4a36-8f31-146fa9cab6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:10.999236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1097/2214] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:11.021305 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98820e-59d8-4dbd-a2c5-cc2e77a64af3 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:11.095461 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98820e-59d8-4dbd-a2c5-cc2e77a64af3 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:11.099396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8a010-904b-45ac-b26f-5863256fd1cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:11.100008 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8a010-904b-45ac-b26f-5863256fd1cb service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:11.100350 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8a010-904b-45ac-b26f-5863256fd1cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:11.100551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8a010-904b-45ac-b26f-5863256fd1cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:11.114777 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da8a010-904b-45ac-b26f-5863256fd1cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:11.116386 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da8a010-904b-45ac-b26f-5863256fd1cb service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:11.139790 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1118/2215] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:38:11.272529 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c010fa-9eac-4a36-8f31-146fa9cab6e5 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:11.273729 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f1c010fa-9eac-4a36-8f31-146fa9cab6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:11.287863 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:11.297477 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c010fa-9eac-4a36-8f31-146fa9cab6e5 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:11.321817 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef72da-102f-4883-acf3-8f41b2715b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:11.322496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef72da-102f-4883-acf3-8f41b2715b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:11.322781 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef72da-102f-4883-acf3-8f41b2715b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:11.323043 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef72da-102f-4883-acf3-8f41b2715b70 admin admin] PATH_INFO: `/v3/projects/7ca261b5add24645a546a7887a67f44b` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:11.324191 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef72da-102f-4883-acf3-8f41b2715b70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ca261b5add24645a546a7887a67f44b)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:11.326375 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef72da-102f-4883-acf3-8f41b2715b70 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:11.431357 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b1ef72da-102f-4883-acf3-8f41b2715b70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ca261b5add24645a546a7887a67f44b', 'request_id': 'req-b1ef72da-102f-4883-acf3-8f41b2715b70'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:11.446907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b1ef72da-102f-4883-acf3-8f41b2715b70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ca261b5add24645a546a7887a67f44b', 'request_id': 'req-b1ef72da-102f-4883-acf3-8f41b2715b70'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:11.462182 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b1ef72da-102f-4883-acf3-8f41b2715b70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ca261b5add24645a546a7887a67f44b', 'request_id': 'req-b1ef72da-102f-4883-acf3-8f41b2715b70'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:11.490702 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b1ef72da-102f-4883-acf3-8f41b2715b70 admin admin] The token cache is being invalidate because project 7ca261b5add24645a546a7887a67f44b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:38:11.491080 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b1ef72da-102f-4883-acf3-8f41b2715b70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:11.492404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b1ef72da-102f-4883-acf3-8f41b2715b70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:11.498077 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1098/2216] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:38:11 2022] DELETE /identity/v3/projects/7ca261b5add24645a546a7887a67f44b => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:11.502219 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8a010-904b-45ac-b26f-5863256fd1cb service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:11.520608 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8a010-904b-45ac-b26f-5863256fd1cb service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:11.651290 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab361f5-dca5-41d2-bbfc-92060211f713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:11.652688 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab361f5-dca5-41d2-bbfc-92060211f713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:11.653102 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab361f5-dca5-41d2-bbfc-92060211f713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:11.653481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab361f5-dca5-41d2-bbfc-92060211f713 admin admin] PATH_INFO: `/v3/users/a3b6dbc3f94d4efa8502cf1b38c9b721` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:11.656514 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab361f5-dca5-41d2-bbfc-92060211f713 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3b6dbc3f94d4efa8502cf1b38c9b721)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:11.659415 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab361f5-dca5-41d2-bbfc-92060211f713 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:11.798749 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-aab361f5-dca5-41d2-bbfc-92060211f713 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3b6dbc3f94d4efa8502cf1b38c9b721'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:11.800284 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-aab361f5-dca5-41d2-bbfc-92060211f713 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3b6dbc3f94d4efa8502cf1b38c9b721'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:11.825862 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-aab361f5-dca5-41d2-bbfc-92060211f713 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3b6dbc3f94d4efa8502cf1b38c9b721'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:11.858269 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:11.864560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-aab361f5-dca5-41d2-bbfc-92060211f713 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3b6dbc3f94d4efa8502cf1b38c9b721'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:11.871920 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-aab361f5-dca5-41d2-bbfc-92060211f713 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3b6dbc3f94d4efa8502cf1b38c9b721'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:11.897012 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1119/2217] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:38:11 2022] DELETE /identity/v3/users/a3b6dbc3f94d4efa8502cf1b38c9b721 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:12.328742 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02846e11-99ec-4f9f-b172-a3ac48732e47 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:12.329239 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02846e11-99ec-4f9f-b172-a3ac48732e47 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:12.329508 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02846e11-99ec-4f9f-b172-a3ac48732e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:12.385014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab361f5-dca5-41d2-bbfc-92060211f713 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:12.386905 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-aab361f5-dca5-41d2-bbfc-92060211f713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:12.397103 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-02846e11-99ec-4f9f-b172-a3ac48732e47 None None] MFA Rules not processed for user `757d987b71484f8aaa24062d6e509aa8`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:12.406645 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-02846e11-99ec-4f9f-b172-a3ac48732e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:12.506596 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab361f5-dca5-41d2-bbfc-92060211f713 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:12.513710 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6e8ed-bc08-4c04-a44e-9d79493d856d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:12.514478 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6e8ed-bc08-4c04-a44e-9d79493d856d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:12.514615 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6e8ed-bc08-4c04-a44e-9d79493d856d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:12.514886 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6e8ed-bc08-4c04-a44e-9d79493d856d admin admin] PATH_INFO: `/v3/projects/94e1183c6b1844188399836f00d61012` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:12.516372 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6e8ed-bc08-4c04-a44e-9d79493d856d admin admin] RBAC: Authorizing `identity:delete_project(project_id=94e1183c6b1844188399836f00d61012)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:12.517983 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6e8ed-bc08-4c04-a44e-9d79493d856d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:12.624535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-eeb6e8ed-bc08-4c04-a44e-9d79493d856d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94e1183c6b1844188399836f00d61012', 'request_id': 'req-eeb6e8ed-bc08-4c04-a44e-9d79493d856d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:12.636758 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1099/2218] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:12 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:12.646054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02846e11-99ec-4f9f-b172-a3ac48732e47 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:12.651948 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-eeb6e8ed-bc08-4c04-a44e-9d79493d856d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94e1183c6b1844188399836f00d61012', 'request_id': 'req-eeb6e8ed-bc08-4c04-a44e-9d79493d856d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:12.653966 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-02846e11-99ec-4f9f-b172-a3ac48732e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:12.666778 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-eeb6e8ed-bc08-4c04-a44e-9d79493d856d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94e1183c6b1844188399836f00d61012', 'request_id': 'req-eeb6e8ed-bc08-4c04-a44e-9d79493d856d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:12.686659 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02846e11-99ec-4f9f-b172-a3ac48732e47 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:12.696582 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-eeb6e8ed-bc08-4c04-a44e-9d79493d856d admin admin] The token cache is being invalidate because project 94e1183c6b1844188399836f00d61012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:38:12.696991 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-eeb6e8ed-bc08-4c04-a44e-9d79493d856d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:12.699803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-eeb6e8ed-bc08-4c04-a44e-9d79493d856d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:12.702279 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1120/2219] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:38:12 2022] DELETE /identity/v3/projects/94e1183c6b1844188399836f00d61012 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:12.707473 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c25bad-8409-400d-9787-8a93a9f58ad4 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:12.707749 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c25bad-8409-400d-9787-8a93a9f58ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:12.708020 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c25bad-8409-400d-9787-8a93a9f58ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:12.748595 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-41c25bad-8409-400d-9787-8a93a9f58ad4 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:12.750083 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-41c25bad-8409-400d-9787-8a93a9f58ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:12.798661 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3adb3a-3aa7-45b7-b19f-dc560e6dfe43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:12.799286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3adb3a-3aa7-45b7-b19f-dc560e6dfe43 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:12.799567 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3adb3a-3aa7-45b7-b19f-dc560e6dfe43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:12.799841 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3adb3a-3aa7-45b7-b19f-dc560e6dfe43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:12.800793 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3adb3a-3aa7-45b7-b19f-dc560e6dfe43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:12.802848 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3adb3a-3aa7-45b7-b19f-dc560e6dfe43 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:12.844411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1121/2220] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:12.845871 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccbf83-3794-4016-baf6-1625903313ff None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:12.846156 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccbf83-3794-4016-baf6-1625903313ff None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:12.846457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccbf83-3794-4016-baf6-1625903313ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:12.868421 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1100/2221] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:38:12 2022] POST /identity/v3/projects => generated 403 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:12.872308 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3adb3a-3aa7-45b7-b19f-dc560e6dfe43 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:12.875912 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1e3adb3a-3aa7-45b7-b19f-dc560e6dfe43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:12.878070 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-89ccbf83-3794-4016-baf6-1625903313ff None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:12.879350 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-89ccbf83-3794-4016-baf6-1625903313ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:12.891065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:12.892411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:12.981745 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3adb3a-3aa7-45b7-b19f-dc560e6dfe43 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:12.987007 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7e7cd-7aad-42bf-bfb2-83ea176cb9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:12.987621 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7e7cd-7aad-42bf-bfb2-83ea176cb9b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:12.987901 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7e7cd-7aad-42bf-bfb2-83ea176cb9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:12.988174 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7e7cd-7aad-42bf-bfb2-83ea176cb9b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:12.988837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7e7cd-7aad-42bf-bfb2-83ea176cb9b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:12.990773 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7e7cd-7aad-42bf-bfb2-83ea176cb9b3 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:13.004013 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1122/2222] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:13.005243 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ccbf83-3794-4016-baf6-1625903313ff None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:13.017939 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ccbf83-3794-4016-baf6-1625903313ff None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:13.020622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1101/2223] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:38:12 2022] POST /identity/v3/users => generated 308 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:13.023687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff90f3-305f-49c0-8629-e4e5c892be21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:13.025387 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff90f3-305f-49c0-8629-e4e5c892be21 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:13.025509 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7e7cd-7aad-42bf-bfb2-83ea176cb9b3 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:13.026397 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff90f3-305f-49c0-8629-e4e5c892be21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:13.026846 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff90f3-305f-49c0-8629-e4e5c892be21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:13.027670 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff90f3-305f-49c0-8629-e4e5c892be21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:38:13.028249 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff90f3-305f-49c0-8629-e4e5c892be21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:13.030309 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff90f3-305f-49c0-8629-e4e5c892be21 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:13.037529 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7e7cd-7aad-42bf-bfb2-83ea176cb9b3 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:13.043165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652201d-1c70-42ce-a72a-a6a84a1f96ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:13.043977 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652201d-1c70-42ce-a72a-a6a84a1f96ff admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:13.044239 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652201d-1c70-42ce-a72a-a6a84a1f96ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:13.044525 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652201d-1c70-42ce-a72a-a6a84a1f96ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:13.045197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2652201d-1c70-42ce-a72a-a6a84a1f96ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:13.050152 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2652201d-1c70-42ce-a72a-a6a84a1f96ff admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:13.050854 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1123/2224] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:38:13.051883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff90f3-305f-49c0-8629-e4e5c892be21 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:13.060578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1102/2225] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:38:13.064067 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff90f3-305f-49c0-8629-e4e5c892be21 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:13.065219 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652201d-1c70-42ce-a72a-a6a84a1f96ff admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:13.068869 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972740be-8ea7-4445-aa11-dfcdcacc13bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:13.069611 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972740be-8ea7-4445-aa11-dfcdcacc13bc admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:13.069611 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972740be-8ea7-4445-aa11-dfcdcacc13bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:13.069860 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972740be-8ea7-4445-aa11-dfcdcacc13bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:13.071570 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972740be-8ea7-4445-aa11-dfcdcacc13bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:13.074357 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972740be-8ea7-4445-aa11-dfcdcacc13bc admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:13.077473 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652201d-1c70-42ce-a72a-a6a84a1f96ff admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:13.082947 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83831e-a3d4-4935-b04b-295323d73733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:13.083835 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83831e-a3d4-4935-b04b-295323d73733 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:13.084294 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83831e-a3d4-4935-b04b-295323d73733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:13.084580 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83831e-a3d4-4935-b04b-295323d73733 admin admin] PATH_INFO: `/v3/projects/6ff2beaac2a6493fb8b5268512e3a8e7/users/ce6d949fe8e047b08f3d07293f75cacf/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:13.107059 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea83831e-a3d4-4935-b04b-295323d73733 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ff2beaac2a6493fb8b5268512e3a8e7, user_id=ce6d949fe8e047b08f3d07293f75cacf, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:13.112865 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1124/2226] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:38:13 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:13.113009 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea83831e-a3d4-4935-b04b-295323d73733 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:13.114200 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972740be-8ea7-4445-aa11-dfcdcacc13bc admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:13.126206 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1103/2227] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:38:13 2022] PUT /identity/v3/projects/6ff2beaac2a6493fb8b5268512e3a8e7/users/ce6d949fe8e047b08f3d07293f75cacf/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:38:13.130735 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83831e-a3d4-4935-b04b-295323d73733 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:13.224091 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972740be-8ea7-4445-aa11-dfcdcacc13bc admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:13.230888 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83831e-a3d4-4935-b04b-295323d73733 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:13.234563 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257694e-5f1e-4b28-bf8f-cc0aba543c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:13.235376 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257694e-5f1e-4b28-bf8f-cc0aba543c5d admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:13.235642 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257694e-5f1e-4b28-bf8f-cc0aba543c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:13.235930 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257694e-5f1e-4b28-bf8f-cc0aba543c5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:13.236866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b257694e-5f1e-4b28-bf8f-cc0aba543c5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:13.237491 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f4488-74b2-49ae-a4d1-1b015e3d0db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:13.238403 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f4488-74b2-49ae-a4d1-1b015e3d0db2 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:13.238562 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b257694e-5f1e-4b28-bf8f-cc0aba543c5d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:13.238658 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f4488-74b2-49ae-a4d1-1b015e3d0db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:13.238929 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f4488-74b2-49ae-a4d1-1b015e3d0db2 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:13.240118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f4488-74b2-49ae-a4d1-1b015e3d0db2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:13.242093 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f4488-74b2-49ae-a4d1-1b015e3d0db2 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:13.250567 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1104/2228] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:38:13.254426 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257694e-5f1e-4b28-bf8f-cc0aba543c5d admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:13.268323 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257694e-5f1e-4b28-bf8f-cc0aba543c5d admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:13.278313 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1125/2229] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:38:13 2022] POST /identity/v3/users => generated 291 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:38:13.281631 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f4488-74b2-49ae-a4d1-1b015e3d0db2 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:13.285003 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4c3c8-dd3a-49f5-831e-3f73786eb300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:13.285936 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4c3c8-dd3a-49f5-831e-3f73786eb300 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:13.286226 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4c3c8-dd3a-49f5-831e-3f73786eb300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:13.286501 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4c3c8-dd3a-49f5-831e-3f73786eb300 admin admin] PATH_INFO: `/v3/projects/6ff2beaac2a6493fb8b5268512e3a8e7/users/ce6d949fe8e047b08f3d07293f75cacf/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:13.293151 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f4488-74b2-49ae-a4d1-1b015e3d0db2 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:13.293440 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4c3c8-dd3a-49f5-831e-3f73786eb300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ff2beaac2a6493fb8b5268512e3a8e7, user_id=ce6d949fe8e047b08f3d07293f75cacf, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:13.298648 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4c3c8-dd3a-49f5-831e-3f73786eb300 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:13.300819 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0fe7c-cf46-4056-af98-ab1971b7b60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:13.301441 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0fe7c-cf46-4056-af98-ab1971b7b60d admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:13.301719 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0fe7c-cf46-4056-af98-ab1971b7b60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:13.301989 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0fe7c-cf46-4056-af98-ab1971b7b60d admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:13.302628 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0fe7c-cf46-4056-af98-ab1971b7b60d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:13.304212 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0fe7c-cf46-4056-af98-ab1971b7b60d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:13.316235 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1105/2230] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:38:13 2022] PUT /identity/v3/projects/6ff2beaac2a6493fb8b5268512e3a8e7/users/ce6d949fe8e047b08f3d07293f75cacf/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:13.317363 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1126/2231] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:38:13.326850 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4c3c8-dd3a-49f5-831e-3f73786eb300 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:13.392221 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:13.449800 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4c3c8-dd3a-49f5-831e-3f73786eb300 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:13.456570 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4683e89c-5356-475a-bf56-983f585355ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:13.457665 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683e89c-5356-475a-bf56-983f585355ef admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:13.457954 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683e89c-5356-475a-bf56-983f585355ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:13.458336 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683e89c-5356-475a-bf56-983f585355ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:13.458992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4683e89c-5356-475a-bf56-983f585355ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:13.462113 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4683e89c-5356-475a-bf56-983f585355ef admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:13.476809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1106/2232] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:38:13.480934 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0fe7c-cf46-4056-af98-ab1971b7b60d admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:13.495510 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0fe7c-cf46-4056-af98-ab1971b7b60d admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:13.506354 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc22d5d-c587-4255-a807-2382d0893b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:13.507175 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc22d5d-c587-4255-a807-2382d0893b5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:13.507558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc22d5d-c587-4255-a807-2382d0893b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:13.509192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc22d5d-c587-4255-a807-2382d0893b5a admin admin] PATH_INFO: `/v3/projects/50afaa2b012f43de81324f9f94d95803/users/e1b86ae422b248829d1189c8013c733a/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:13.540657 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc22d5d-c587-4255-a807-2382d0893b5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=50afaa2b012f43de81324f9f94d95803, user_id=e1b86ae422b248829d1189c8013c733a, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:13.543619 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc22d5d-c587-4255-a807-2382d0893b5a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:13.568961 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1127/2233] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:38:13 2022] PUT /identity/v3/projects/50afaa2b012f43de81324f9f94d95803/users/e1b86ae422b248829d1189c8013c733a/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:38:13.577458 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df6f53-cbbb-40d7-ab1d-ab7367144fc9 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:13.577797 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df6f53-cbbb-40d7-ab1d-ab7367144fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:13.578082 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df6f53-cbbb-40d7-ab1d-ab7367144fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:13.618119 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-33df6f53-cbbb-40d7-ab1d-ab7367144fc9 None None] MFA Rules not processed for user `e1b86ae422b248829d1189c8013c733a`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:13.619359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-33df6f53-cbbb-40d7-ab1d-ab7367144fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:13.730896 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1107/2234] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:13.736139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc22d5d-c587-4255-a807-2382d0893b5a admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:13.806807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:13.864006 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc22d5d-c587-4255-a807-2382d0893b5a admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:13.870430 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966a971-1ce9-4ff1-b62a-e58b0948ef27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:13.871243 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966a971-1ce9-4ff1-b62a-e58b0948ef27 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:13.871825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966a971-1ce9-4ff1-b62a-e58b0948ef27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:13.872426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966a971-1ce9-4ff1-b62a-e58b0948ef27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:13.873492 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966a971-1ce9-4ff1-b62a-e58b0948ef27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:13.875809 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966a971-1ce9-4ff1-b62a-e58b0948ef27 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:13.919789 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1128/2235] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:38:13 2022] POST /identity/v3/projects => generated 383 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:13.923660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df6f53-cbbb-40d7-ab1d-ab7367144fc9 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:13.949894 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df6f53-cbbb-40d7-ab1d-ab7367144fc9 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:14.084475 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99e1c5-c70f-4125-aba5-c69c23447a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:14.085151 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99e1c5-c70f-4125-aba5-c69c23447a45 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:14.085432 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99e1c5-c70f-4125-aba5-c69c23447a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:14.085762 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99e1c5-c70f-4125-aba5-c69c23447a45 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:14.086452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a99e1c5-c70f-4125-aba5-c69c23447a45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:14.089283 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a99e1c5-c70f-4125-aba5-c69c23447a45 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:14.131624 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1108/2236] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:38:13 2022] POST /identity/v3/users => generated 298 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:14.137263 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966a971-1ce9-4ff1-b62a-e58b0948ef27 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:14.151219 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966a971-1ce9-4ff1-b62a-e58b0948ef27 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:14.160898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ed298-34c2-48ca-9e8b-c4058a27da5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:14.162117 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ed298-34c2-48ca-9e8b-c4058a27da5c admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:14.163717 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ed298-34c2-48ca-9e8b-c4058a27da5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:14.164100 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ed298-34c2-48ca-9e8b-c4058a27da5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:14.165064 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ed298-34c2-48ca-9e8b-c4058a27da5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:14.166799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ed298-34c2-48ca-9e8b-c4058a27da5c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:14.187292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1129/2237] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:38:14.189871 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99e1c5-c70f-4125-aba5-c69c23447a45 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:14.215515 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99e1c5-c70f-4125-aba5-c69c23447a45 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:14.222428 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468af9f9-058f-41fa-94c4-4b3dfa9c305e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:14.223104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468af9f9-058f-41fa-94c4-4b3dfa9c305e admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:14.223381 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468af9f9-058f-41fa-94c4-4b3dfa9c305e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:14.223660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468af9f9-058f-41fa-94c4-4b3dfa9c305e admin admin] PATH_INFO: `/v3/projects/87df72e1192a4578b770785dc5ed12fc/users/9c568917c053466684d64ad853fc08ff/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:14.250980 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc35a90-2641-4c2e-8568-3bb94fe575eb None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:14.251394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc35a90-2641-4c2e-8568-3bb94fe575eb None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:14.251775 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc35a90-2641-4c2e-8568-3bb94fe575eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:14.257648 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468af9f9-058f-41fa-94c4-4b3dfa9c305e admin admin] RBAC: Authorizing `identity:create_grant(project_id=87df72e1192a4578b770785dc5ed12fc, user_id=9c568917c053466684d64ad853fc08ff, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:14.261806 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468af9f9-058f-41fa-94c4-4b3dfa9c305e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:14.290000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1109/2238] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:38:14 2022] PUT /identity/v3/projects/87df72e1192a4578b770785dc5ed12fc/users/9c568917c053466684d64ad853fc08ff/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:38:14.293455 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468af9f9-058f-41fa-94c4-4b3dfa9c305e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:14.308390 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-7bc35a90-2641-4c2e-8568-3bb94fe575eb None None] MFA Rules not processed for user `ce6d949fe8e047b08f3d07293f75cacf`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:14.312204 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7bc35a90-2641-4c2e-8568-3bb94fe575eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:14.384199 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:14.441184 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468af9f9-058f-41fa-94c4-4b3dfa9c305e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:14.447978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a9628-cbc3-46d1-9040-0480bc5775eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:14.449178 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a9628-cbc3-46d1-9040-0480bc5775eb admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:14.449466 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a9628-cbc3-46d1-9040-0480bc5775eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:14.449747 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a9628-cbc3-46d1-9040-0480bc5775eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:14.453028 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a9628-cbc3-46d1-9040-0480bc5775eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:14.455792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a9628-cbc3-46d1-9040-0480bc5775eb admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:14.476297 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1110/2239] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:38:14.489226 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a9628-cbc3-46d1-9040-0480bc5775eb admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:14.496111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1130/2240] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:14 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:14.507547 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a9628-cbc3-46d1-9040-0480bc5775eb admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:14.514111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235fb953-f85f-4f63-bb92-6b6e2bd1b1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:14.515114 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fb953-f85f-4f63-bb92-6b6e2bd1b1e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:14.515584 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fb953-f85f-4f63-bb92-6b6e2bd1b1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:14.516006 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fb953-f85f-4f63-bb92-6b6e2bd1b1e1 admin admin] PATH_INFO: `/v3/projects/87df72e1192a4578b770785dc5ed12fc/users/9c568917c053466684d64ad853fc08ff/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:14.518196 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235fb953-f85f-4f63-bb92-6b6e2bd1b1e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87df72e1192a4578b770785dc5ed12fc, user_id=9c568917c053466684d64ad853fc08ff, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:14.521398 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235fb953-f85f-4f63-bb92-6b6e2bd1b1e1 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:14.534107 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1111/2241] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:38:14 2022] PUT /identity/v3/projects/87df72e1192a4578b770785dc5ed12fc/users/9c568917c053466684d64ad853fc08ff/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:14.544839 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500827da-9386-4e82-aee0-8c41dce473b3 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:14.545122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500827da-9386-4e82-aee0-8c41dce473b3 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:14.545396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500827da-9386-4e82-aee0-8c41dce473b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:14.589369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-500827da-9386-4e82-aee0-8c41dce473b3 None None] MFA Rules not processed for user `9c568917c053466684d64ad853fc08ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:14.590636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-500827da-9386-4e82-aee0-8c41dce473b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:14.672247 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1131/2242] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:38:14.685801 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235fb953-f85f-4f63-bb92-6b6e2bd1b1e1 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:14.688339 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-235fb953-f85f-4f63-bb92-6b6e2bd1b1e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:14.780785 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235fb953-f85f-4f63-bb92-6b6e2bd1b1e1 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:14.785107 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12062e5-7800-4a55-8b35-b4f59f734d0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:14.785801 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12062e5-7800-4a55-8b35-b4f59f734d0e service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:14.786443 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12062e5-7800-4a55-8b35-b4f59f734d0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:14.786443 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12062e5-7800-4a55-8b35-b4f59f734d0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:14.887163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12062e5-7800-4a55-8b35-b4f59f734d0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:14.889333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12062e5-7800-4a55-8b35-b4f59f734d0e service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:14.901797 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1112/2243] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:38:15.811254 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500827da-9386-4e82-aee0-8c41dce473b3 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:15.840407 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:15.891887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500827da-9386-4e82-aee0-8c41dce473b3 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:15.896570 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d45440-e529-4193-9da5-c30d62c6ea05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:15.907856 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d45440-e529-4193-9da5-c30d62c6ea05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:15.908144 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d45440-e529-4193-9da5-c30d62c6ea05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:15.908425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d45440-e529-4193-9da5-c30d62c6ea05 admin admin] PATH_INFO: `/v3/users/757d987b71484f8aaa24062d6e509aa8` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:15.909757 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d45440-e529-4193-9da5-c30d62c6ea05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=757d987b71484f8aaa24062d6e509aa8)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:15.911693 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d45440-e529-4193-9da5-c30d62c6ea05 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:15.970124 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-88d45440-e529-4193-9da5-c30d62c6ea05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '757d987b71484f8aaa24062d6e509aa8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:15.970989 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-88d45440-e529-4193-9da5-c30d62c6ea05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '757d987b71484f8aaa24062d6e509aa8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:15.981069 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-88d45440-e529-4193-9da5-c30d62c6ea05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '757d987b71484f8aaa24062d6e509aa8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:15.995000 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-88d45440-e529-4193-9da5-c30d62c6ea05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '757d987b71484f8aaa24062d6e509aa8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:15.995681 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-88d45440-e529-4193-9da5-c30d62c6ea05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '757d987b71484f8aaa24062d6e509aa8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:16.011906 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1132/2244] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:38:15 2022] DELETE /identity/v3/users/757d987b71484f8aaa24062d6e509aa8 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:16.219410 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12062e5-7800-4a55-8b35-b4f59f734d0e service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:16.234347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12062e5-7800-4a55-8b35-b4f59f734d0e service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:16.308488 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:16.316079 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18d8b4-507d-4490-9374-8379ac142601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:16.317206 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18d8b4-507d-4490-9374-8379ac142601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:16.317586 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18d8b4-507d-4490-9374-8379ac142601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:16.317964 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18d8b4-507d-4490-9374-8379ac142601 admin admin] PATH_INFO: `/v3/projects/41866630bfff4e31b10bbde20a42290b` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:16.321971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd18d8b4-507d-4490-9374-8379ac142601 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41866630bfff4e31b10bbde20a42290b)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:16.321971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd18d8b4-507d-4490-9374-8379ac142601 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:16.374416 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fd18d8b4-507d-4490-9374-8379ac142601 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41866630bfff4e31b10bbde20a42290b', 'request_id': 'req-fd18d8b4-507d-4490-9374-8379ac142601'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:16.386312 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fd18d8b4-507d-4490-9374-8379ac142601 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41866630bfff4e31b10bbde20a42290b', 'request_id': 'req-fd18d8b4-507d-4490-9374-8379ac142601'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:16.393252 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fd18d8b4-507d-4490-9374-8379ac142601 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41866630bfff4e31b10bbde20a42290b', 'request_id': 'req-fd18d8b4-507d-4490-9374-8379ac142601'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:16.409609 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fd18d8b4-507d-4490-9374-8379ac142601 admin admin] The token cache is being invalidate because project 41866630bfff4e31b10bbde20a42290b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:38:16.409899 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fd18d8b4-507d-4490-9374-8379ac142601 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:16.410563 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fd18d8b4-507d-4490-9374-8379ac142601 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:16.412250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1113/2245] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:38:16 2022] DELETE /identity/v3/projects/41866630bfff4e31b10bbde20a42290b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:16.417435 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d45440-e529-4193-9da5-c30d62c6ea05 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:16.418870 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-88d45440-e529-4193-9da5-c30d62c6ea05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:16.432395 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d45440-e529-4193-9da5-c30d62c6ea05 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:16.529575 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce04587-0a1c-4aef-91b1-2aa405de4be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:16.530247 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce04587-0a1c-4aef-91b1-2aa405de4be1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:16.530524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce04587-0a1c-4aef-91b1-2aa405de4be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:16.530795 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce04587-0a1c-4aef-91b1-2aa405de4be1 admin admin] PATH_INFO: `/v3/users/ce6d949fe8e047b08f3d07293f75cacf` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:16.531983 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce04587-0a1c-4aef-91b1-2aa405de4be1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce6d949fe8e047b08f3d07293f75cacf)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:16.534715 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce04587-0a1c-4aef-91b1-2aa405de4be1 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:16.627396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:16.652328 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9ce04587-0a1c-4aef-91b1-2aa405de4be1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce6d949fe8e047b08f3d07293f75cacf'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:16.652328 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9ce04587-0a1c-4aef-91b1-2aa405de4be1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce6d949fe8e047b08f3d07293f75cacf'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:16.668307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9ce04587-0a1c-4aef-91b1-2aa405de4be1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce6d949fe8e047b08f3d07293f75cacf'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:16.690151 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9ce04587-0a1c-4aef-91b1-2aa405de4be1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce6d949fe8e047b08f3d07293f75cacf'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:16.691311 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9ce04587-0a1c-4aef-91b1-2aa405de4be1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce6d949fe8e047b08f3d07293f75cacf'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:16.706262 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1133/2246] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:38:16 2022] DELETE /identity/v3/users/ce6d949fe8e047b08f3d07293f75cacf => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:16.959252 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18d8b4-507d-4490-9374-8379ac142601 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:16.962324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fd18d8b4-507d-4490-9374-8379ac142601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:16.979601 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18d8b4-507d-4490-9374-8379ac142601 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:17.081618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582f1be-07cf-4b20-994c-c01efa69faa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:17.082703 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582f1be-07cf-4b20-994c-c01efa69faa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:17.083161 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582f1be-07cf-4b20-994c-c01efa69faa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:17.083440 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582f1be-07cf-4b20-994c-c01efa69faa7 admin admin] PATH_INFO: `/v3/projects/6ff2beaac2a6493fb8b5268512e3a8e7` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:17.084591 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7582f1be-07cf-4b20-994c-c01efa69faa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ff2beaac2a6493fb8b5268512e3a8e7)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:17.086595 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7582f1be-07cf-4b20-994c-c01efa69faa7 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:17.139702 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7582f1be-07cf-4b20-994c-c01efa69faa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ff2beaac2a6493fb8b5268512e3a8e7', 'request_id': 'req-7582f1be-07cf-4b20-994c-c01efa69faa7'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:17.146553 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7582f1be-07cf-4b20-994c-c01efa69faa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ff2beaac2a6493fb8b5268512e3a8e7', 'request_id': 'req-7582f1be-07cf-4b20-994c-c01efa69faa7'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:17.153047 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7582f1be-07cf-4b20-994c-c01efa69faa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ff2beaac2a6493fb8b5268512e3a8e7', 'request_id': 'req-7582f1be-07cf-4b20-994c-c01efa69faa7'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:17.171612 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7582f1be-07cf-4b20-994c-c01efa69faa7 admin admin] The token cache is being invalidate because project 6ff2beaac2a6493fb8b5268512e3a8e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:38:17.171893 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7582f1be-07cf-4b20-994c-c01efa69faa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:17.172662 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7582f1be-07cf-4b20-994c-c01efa69faa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:17.174677 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1114/2247] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:38:16 2022] DELETE /identity/v3/projects/6ff2beaac2a6493fb8b5268512e3a8e7 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:17.181113 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b1285-e3a8-4183-b1b8-8026575721e6 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:17.181646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b1285-e3a8-4183-b1b8-8026575721e6 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:17.181738 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b1285-e3a8-4183-b1b8-8026575721e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:17.207061 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-984b1285-e3a8-4183-b1b8-8026575721e6 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:17.208262 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-984b1285-e3a8-4183-b1b8-8026575721e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:17.301940 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1134/2248] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:17.309846 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4dcc5-1dcf-4f3a-bde4-3632892e3267 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:17.310276 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4dcc5-1dcf-4f3a-bde4-3632892e3267 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:17.310597 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4dcc5-1dcf-4f3a-bde4-3632892e3267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:17.336597 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-a7f4dcc5-1dcf-4f3a-bde4-3632892e3267 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:17.338138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a7f4dcc5-1dcf-4f3a-bde4-3632892e3267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:17.349888 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1115/2249] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:17.351133 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b1285-e3a8-4183-b1b8-8026575721e6 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:17.361933 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b1285-e3a8-4183-b1b8-8026575721e6 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:17.366905 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd6c1d-cbf7-44d8-b666-b46130864403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:17.368160 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd6c1d-cbf7-44d8-b666-b46130864403 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:17.368535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd6c1d-cbf7-44d8-b666-b46130864403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:17.368905 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd6c1d-cbf7-44d8-b666-b46130864403 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:17.369622 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd6c1d-cbf7-44d8-b666-b46130864403 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:38:17.369997 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd6c1d-cbf7-44d8-b666-b46130864403 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:17.371799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd6c1d-cbf7-44d8-b666-b46130864403 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:17.387451 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1135/2250] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:38:17.390742 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4dcc5-1dcf-4f3a-bde4-3632892e3267 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:17.402643 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4dcc5-1dcf-4f3a-bde4-3632892e3267 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:17.407420 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c503b3-734d-48c5-a816-1e7a47a2d7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:17.408030 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c503b3-734d-48c5-a816-1e7a47a2d7bc admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:17.408312 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c503b3-734d-48c5-a816-1e7a47a2d7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:17.408596 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c503b3-734d-48c5-a816-1e7a47a2d7bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:17.409381 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c503b3-734d-48c5-a816-1e7a47a2d7bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:17.412063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c503b3-734d-48c5-a816-1e7a47a2d7bc admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:17.425109 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:17.450543 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1116/2251] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:38:17 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:17.454738 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd6c1d-cbf7-44d8-b666-b46130864403 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:17.467217 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd6c1d-cbf7-44d8-b666-b46130864403 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:17.538953 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e6c28-9988-44ba-820c-6003a7c58e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:17.540089 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e6c28-9988-44ba-820c-6003a7c58e77 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:17.540364 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e6c28-9988-44ba-820c-6003a7c58e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:17.540630 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e6c28-9988-44ba-820c-6003a7c58e77 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:17.541282 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e6c28-9988-44ba-820c-6003a7c58e77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:17.543197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e6c28-9988-44ba-820c-6003a7c58e77 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:17.571076 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1136/2252] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:38:17 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:17.572673 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c503b3-734d-48c5-a816-1e7a47a2d7bc admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:17.587477 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c503b3-734d-48c5-a816-1e7a47a2d7bc admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:17.593376 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e51fa-c139-4280-9efd-a658fac40b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:17.593376 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e51fa-c139-4280-9efd-a658fac40b2e admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:17.593376 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e51fa-c139-4280-9efd-a658fac40b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:17.594330 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e51fa-c139-4280-9efd-a658fac40b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:17.594822 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e51fa-c139-4280-9efd-a658fac40b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:17.598353 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e51fa-c139-4280-9efd-a658fac40b2e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:17.610372 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1117/2253] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:38:17.614219 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e6c28-9988-44ba-820c-6003a7c58e77 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:17.625033 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e6c28-9988-44ba-820c-6003a7c58e77 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:17.635250 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881288ce-bb81-43e3-b6f7-d16c2bce829f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:17.635879 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881288ce-bb81-43e3-b6f7-d16c2bce829f admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:17.636164 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881288ce-bb81-43e3-b6f7-d16c2bce829f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:17.636439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881288ce-bb81-43e3-b6f7-d16c2bce829f admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:17.637119 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881288ce-bb81-43e3-b6f7-d16c2bce829f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:17.638727 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881288ce-bb81-43e3-b6f7-d16c2bce829f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:17.654815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1137/2254] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:38:17.656490 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e51fa-c139-4280-9efd-a658fac40b2e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:17.671912 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e51fa-c139-4280-9efd-a658fac40b2e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:17.676955 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3fb14-cc0e-4f72-b897-325490d99785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:17.677811 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3fb14-cc0e-4f72-b897-325490d99785 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:17.678186 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3fb14-cc0e-4f72-b897-325490d99785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:17.678570 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3fb14-cc0e-4f72-b897-325490d99785 admin admin] PATH_INFO: `/v3/projects/b559bbad3eb047d29aa0ba9be3f73b3b/users/65a3fb4e4af74205a8650f433f99de89/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:17.705284 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3fb14-cc0e-4f72-b897-325490d99785 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b559bbad3eb047d29aa0ba9be3f73b3b, user_id=65a3fb4e4af74205a8650f433f99de89, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:17.708413 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3fb14-cc0e-4f72-b897-325490d99785 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:17.732358 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1118/2255] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:38:17 2022] PUT /identity/v3/projects/b559bbad3eb047d29aa0ba9be3f73b3b/users/65a3fb4e4af74205a8650f433f99de89/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:17.744074 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b03a9-0538-4e7f-8893-9f4687a45764 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:17.744355 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b03a9-0538-4e7f-8893-9f4687a45764 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:17.744631 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b03a9-0538-4e7f-8893-9f4687a45764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:17.775976 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-f14b03a9-0538-4e7f-8893-9f4687a45764 None None] MFA Rules not processed for user `65a3fb4e4af74205a8650f433f99de89`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:17.777459 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f14b03a9-0538-4e7f-8893-9f4687a45764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:17.788226 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:17.860783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1138/2256] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:38:17.862787 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3fb14-cc0e-4f72-b897-325490d99785 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:17.936071 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3fb14-cc0e-4f72-b897-325490d99785 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:17.940829 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2564d-478d-4939-a8bc-bd967ef67cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:17.941449 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2564d-478d-4939-a8bc-bd967ef67cea admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:17.941727 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2564d-478d-4939-a8bc-bd967ef67cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:17.941997 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2564d-478d-4939-a8bc-bd967ef67cea admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:17.942644 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2564d-478d-4939-a8bc-bd967ef67cea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:17.944546 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2564d-478d-4939-a8bc-bd967ef67cea admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:17.975647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1119/2257] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:38:17 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:17.980456 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b03a9-0538-4e7f-8893-9f4687a45764 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:17.991841 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b03a9-0538-4e7f-8893-9f4687a45764 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:18.101463 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f97ff-3253-4408-938d-1fa4bf15ecb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:18.102424 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f97ff-3253-4408-938d-1fa4bf15ecb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:18.102928 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f97ff-3253-4408-938d-1fa4bf15ecb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:18.103280 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f97ff-3253-4408-938d-1fa4bf15ecb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:18.104198 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f97ff-3253-4408-938d-1fa4bf15ecb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:18.107205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f97ff-3253-4408-938d-1fa4bf15ecb4 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:18.148226 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1139/2258] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:38:17 2022] POST /identity/v3/users => generated 299 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:38:18.149017 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2564d-478d-4939-a8bc-bd967ef67cea admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:18.165752 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2564d-478d-4939-a8bc-bd967ef67cea admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:18.171218 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec12549-070f-4727-9b8f-1160d5af948a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:18.171886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec12549-070f-4727-9b8f-1160d5af948a admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:18.172165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec12549-070f-4727-9b8f-1160d5af948a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:18.172435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec12549-070f-4727-9b8f-1160d5af948a admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:18.173158 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec12549-070f-4727-9b8f-1160d5af948a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:18.174872 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec12549-070f-4727-9b8f-1160d5af948a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:18.191487 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1120/2259] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:38:18.196116 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f97ff-3253-4408-938d-1fa4bf15ecb4 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:18.212407 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f97ff-3253-4408-938d-1fa4bf15ecb4 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:18.228885 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b29a80-c614-46f9-98c2-1fb2d7c7f156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:18.229705 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b29a80-c614-46f9-98c2-1fb2d7c7f156 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:18.229990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b29a80-c614-46f9-98c2-1fb2d7c7f156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:18.230268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b29a80-c614-46f9-98c2-1fb2d7c7f156 admin admin] PATH_INFO: `/v3/projects/29ae56a95df04391a79b8dc322cd67bc/users/bfa1343b2bca4506be699bb263767b85/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:18.251237 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b29a80-c614-46f9-98c2-1fb2d7c7f156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29ae56a95df04391a79b8dc322cd67bc, user_id=bfa1343b2bca4506be699bb263767b85, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:18.254686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b29a80-c614-46f9-98c2-1fb2d7c7f156 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:18.280558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1140/2260] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:38:18 2022] PUT /identity/v3/projects/29ae56a95df04391a79b8dc322cd67bc/users/bfa1343b2bca4506be699bb263767b85/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:38:18.281761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec12549-070f-4727-9b8f-1160d5af948a admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:18.311034 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec12549-070f-4727-9b8f-1160d5af948a admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:18.419147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe5338-85fc-443e-9ae6-1630d2e2b317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:18.419859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe5338-85fc-443e-9ae6-1630d2e2b317 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:18.420252 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe5338-85fc-443e-9ae6-1630d2e2b317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:18.420603 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe5338-85fc-443e-9ae6-1630d2e2b317 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:18.421343 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe5338-85fc-443e-9ae6-1630d2e2b317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:18.423138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe5338-85fc-443e-9ae6-1630d2e2b317 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:18.442617 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1121/2261] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:38:18.442891 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b29a80-c614-46f9-98c2-1fb2d7c7f156 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:18.456745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b29a80-c614-46f9-98c2-1fb2d7c7f156 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:18.464298 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f71e7-a867-46ba-9664-bc5dc3691246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:18.465145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f71e7-a867-46ba-9664-bc5dc3691246 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:18.465529 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f71e7-a867-46ba-9664-bc5dc3691246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:18.466527 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f71e7-a867-46ba-9664-bc5dc3691246 admin admin] PATH_INFO: `/v3/projects/29ae56a95df04391a79b8dc322cd67bc/users/bfa1343b2bca4506be699bb263767b85/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:18.468861 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f71e7-a867-46ba-9664-bc5dc3691246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29ae56a95df04391a79b8dc322cd67bc, user_id=bfa1343b2bca4506be699bb263767b85, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:18.472321 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f71e7-a867-46ba-9664-bc5dc3691246 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:18.504788 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1141/2262] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:38:18 2022] PUT /identity/v3/projects/29ae56a95df04391a79b8dc322cd67bc/users/bfa1343b2bca4506be699bb263767b85/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:18.504976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eaee9a-21f5-4db0-9b9f-d02f27868dbc None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:18.504976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eaee9a-21f5-4db0-9b9f-d02f27868dbc None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:18.504976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eaee9a-21f5-4db0-9b9f-d02f27868dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:18.550689 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-40eaee9a-21f5-4db0-9b9f-d02f27868dbc None None] MFA Rules not processed for user `bfa1343b2bca4506be699bb263767b85`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:18.552290 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-40eaee9a-21f5-4db0-9b9f-d02f27868dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:18.583025 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:18.664250 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1122/2263] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:18 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:18.677019 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f71e7-a867-46ba-9664-bc5dc3691246 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:18.679197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f16f71e7-a867-46ba-9664-bc5dc3691246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:18.748558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f71e7-a867-46ba-9664-bc5dc3691246 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:18.758974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351368d-02e9-427a-b279-180d046f90b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:18.759601 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351368d-02e9-427a-b279-180d046f90b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:18.759887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351368d-02e9-427a-b279-180d046f90b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:18.760156 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351368d-02e9-427a-b279-180d046f90b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:18.770838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351368d-02e9-427a-b279-180d046f90b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:18.772359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351368d-02e9-427a-b279-180d046f90b9 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:18.796532 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1142/2264] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:18 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:38:18.817134 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eaee9a-21f5-4db0-9b9f-d02f27868dbc None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:18.902433 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eaee9a-21f5-4db0-9b9f-d02f27868dbc None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:18.907232 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff6a27-2ec7-4e95-ad45-30cd0889d076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:18.908163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff6a27-2ec7-4e95-ad45-30cd0889d076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:18.908626 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff6a27-2ec7-4e95-ad45-30cd0889d076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:18.908992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff6a27-2ec7-4e95-ad45-30cd0889d076 admin admin] PATH_INFO: `/v3/users/65a3fb4e4af74205a8650f433f99de89` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:18.910473 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff6a27-2ec7-4e95-ad45-30cd0889d076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65a3fb4e4af74205a8650f433f99de89)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:18.912556 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff6a27-2ec7-4e95-ad45-30cd0889d076 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:18.979231 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-79ff6a27-2ec7-4e95-ad45-30cd0889d076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65a3fb4e4af74205a8650f433f99de89'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:18.980162 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-79ff6a27-2ec7-4e95-ad45-30cd0889d076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65a3fb4e4af74205a8650f433f99de89'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:18.989126 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-79ff6a27-2ec7-4e95-ad45-30cd0889d076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65a3fb4e4af74205a8650f433f99de89'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:19.005830 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-79ff6a27-2ec7-4e95-ad45-30cd0889d076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65a3fb4e4af74205a8650f433f99de89'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:19.014747 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-79ff6a27-2ec7-4e95-ad45-30cd0889d076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65a3fb4e4af74205a8650f433f99de89'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:19.025219 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1123/2265] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:38:18 2022] DELETE /identity/v3/users/65a3fb4e4af74205a8650f433f99de89 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:19.029491 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0e80e-b586-4304-854a-71322e181ed6 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:19.029844 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0e80e-b586-4304-854a-71322e181ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:19.030187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0e80e-b586-4304-854a-71322e181ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:19.046722 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:19.053268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-c9a0e80e-b586-4304-854a-71322e181ed6 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:19.055230 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c9a0e80e-b586-4304-854a-71322e181ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:19.157213 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1143/2266] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:19.163154 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff6a27-2ec7-4e95-ad45-30cd0889d076 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:19.165216 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-79ff6a27-2ec7-4e95-ad45-30cd0889d076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:19.204187 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:19.244512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff6a27-2ec7-4e95-ad45-30cd0889d076 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:19.253218 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed96a9a4-6980-4e57-be35-5ed55944a632 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:19.254015 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96a9a4-6980-4e57-be35-5ed55944a632 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:19.254360 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96a9a4-6980-4e57-be35-5ed55944a632 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:19.254705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96a9a4-6980-4e57-be35-5ed55944a632 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:19.266192 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed96a9a4-6980-4e57-be35-5ed55944a632 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:19.267633 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed96a9a4-6980-4e57-be35-5ed55944a632 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:19.282029 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1124/2267] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:38:19.663540 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0e80e-b586-4304-854a-71322e181ed6 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:19.666354 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c9a0e80e-b586-4304-854a-71322e181ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:19.680458 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0e80e-b586-4304-854a-71322e181ed6 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:19.686731 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeed518-6bf5-4ee9-ae0c-e85f32ec4fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:19.687467 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeed518-6bf5-4ee9-ae0c-e85f32ec4fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:19.687803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeed518-6bf5-4ee9-ae0c-e85f32ec4fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:19.688177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeed518-6bf5-4ee9-ae0c-e85f32ec4fac admin admin] PATH_INFO: `/v3/projects/b559bbad3eb047d29aa0ba9be3f73b3b` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:19.689366 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeed518-6bf5-4ee9-ae0c-e85f32ec4fac admin admin] RBAC: Authorizing `identity:delete_project(project_id=b559bbad3eb047d29aa0ba9be3f73b3b)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:19.691680 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeed518-6bf5-4ee9-ae0c-e85f32ec4fac admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:19.719599 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed96a9a4-6980-4e57-be35-5ed55944a632 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:19.721774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ed96a9a4-6980-4e57-be35-5ed55944a632 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:19.750170 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-edeed518-6bf5-4ee9-ae0c-e85f32ec4fac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b559bbad3eb047d29aa0ba9be3f73b3b', 'request_id': 'req-edeed518-6bf5-4ee9-ae0c-e85f32ec4fac'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:19.759585 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-edeed518-6bf5-4ee9-ae0c-e85f32ec4fac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b559bbad3eb047d29aa0ba9be3f73b3b', 'request_id': 'req-edeed518-6bf5-4ee9-ae0c-e85f32ec4fac'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:19.767318 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-edeed518-6bf5-4ee9-ae0c-e85f32ec4fac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b559bbad3eb047d29aa0ba9be3f73b3b', 'request_id': 'req-edeed518-6bf5-4ee9-ae0c-e85f32ec4fac'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:19.789917 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-edeed518-6bf5-4ee9-ae0c-e85f32ec4fac admin admin] The token cache is being invalidate because project b559bbad3eb047d29aa0ba9be3f73b3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:38:19.790210 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-edeed518-6bf5-4ee9-ae0c-e85f32ec4fac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:19.790890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-edeed518-6bf5-4ee9-ae0c-e85f32ec4fac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:19.792486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1144/2268] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:38:19 2022] DELETE /identity/v3/projects/b559bbad3eb047d29aa0ba9be3f73b3b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:19.796631 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeed518-6bf5-4ee9-ae0c-e85f32ec4fac admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:19.798836 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-edeed518-6bf5-4ee9-ae0c-e85f32ec4fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:19.801798 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed96a9a4-6980-4e57-be35-5ed55944a632 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:19.889506 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0932ed6-a74a-4c44-9624-e24019592632 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:19.890351 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0932ed6-a74a-4c44-9624-e24019592632 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:19.890686 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0932ed6-a74a-4c44-9624-e24019592632 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:19.891009 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0932ed6-a74a-4c44-9624-e24019592632 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:19.891124 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeed518-6bf5-4ee9-ae0c-e85f32ec4fac admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:19.893271 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b0932ed6-a74a-4c44-9624-e24019592632 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:19.896618 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2699da-03bc-4f72-b7bd-997893e3e778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:19.897329 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2699da-03bc-4f72-b7bd-997893e3e778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:19.897619 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2699da-03bc-4f72-b7bd-997893e3e778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:19.897900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2699da-03bc-4f72-b7bd-997893e3e778 admin admin] PATH_INFO: `/v3/users/bfa1343b2bca4506be699bb263767b85` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:19.900242 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2699da-03bc-4f72-b7bd-997893e3e778 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfa1343b2bca4506be699bb263767b85)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:19.902213 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2699da-03bc-4f72-b7bd-997893e3e778 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:19.948585 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:19.972437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cd2699da-03bc-4f72-b7bd-997893e3e778 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfa1343b2bca4506be699bb263767b85'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:19.974036 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cd2699da-03bc-4f72-b7bd-997893e3e778 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfa1343b2bca4506be699bb263767b85'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:19.985659 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cd2699da-03bc-4f72-b7bd-997893e3e778 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfa1343b2bca4506be699bb263767b85'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:19.989743 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0932ed6-a74a-4c44-9624-e24019592632 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:19.991332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0932ed6-a74a-4c44-9624-e24019592632 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:20.003328 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1125/2269] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:38:20.008893 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cd2699da-03bc-4f72-b7bd-997893e3e778 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfa1343b2bca4506be699bb263767b85'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:20.009611 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cd2699da-03bc-4f72-b7bd-997893e3e778 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfa1343b2bca4506be699bb263767b85'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:20.031385 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1145/2270] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:38:19 2022] DELETE /identity/v3/users/bfa1343b2bca4506be699bb263767b85 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:20.384616 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0932ed6-a74a-4c44-9624-e24019592632 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:20.399641 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0932ed6-a74a-4c44-9624-e24019592632 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:20.465078 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:20.500223 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2be2ae-b598-4168-b69a-4ed77401a368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:20.501056 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2be2ae-b598-4168-b69a-4ed77401a368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:20.501408 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2be2ae-b598-4168-b69a-4ed77401a368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:20.501854 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2be2ae-b598-4168-b69a-4ed77401a368 admin admin] PATH_INFO: `/v3/projects/29ae56a95df04391a79b8dc322cd67bc` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:20.503217 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2be2ae-b598-4168-b69a-4ed77401a368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29ae56a95df04391a79b8dc322cd67bc)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:20.505813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2be2ae-b598-4168-b69a-4ed77401a368 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:20.552137 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe399484-69cc-43b0-99da-c3387525df50 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:20.552427 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe399484-69cc-43b0-99da-c3387525df50 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:20.552724 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe399484-69cc-43b0-99da-c3387525df50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:20.563263 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1c2be2ae-b598-4168-b69a-4ed77401a368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29ae56a95df04391a79b8dc322cd67bc', 'request_id': 'req-1c2be2ae-b598-4168-b69a-4ed77401a368'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:20.569067 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1c2be2ae-b598-4168-b69a-4ed77401a368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29ae56a95df04391a79b8dc322cd67bc', 'request_id': 'req-1c2be2ae-b598-4168-b69a-4ed77401a368'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:20.574283 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-fe399484-69cc-43b0-99da-c3387525df50 None None] MFA Rules not processed for user `30be578ddd664bc79498d9a21f708f04`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:20.575495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-fe399484-69cc-43b0-99da-c3387525df50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:20.575610 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1c2be2ae-b598-4168-b69a-4ed77401a368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29ae56a95df04391a79b8dc322cd67bc', 'request_id': 'req-1c2be2ae-b598-4168-b69a-4ed77401a368'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:20.590172 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1c2be2ae-b598-4168-b69a-4ed77401a368 admin admin] The token cache is being invalidate because project 29ae56a95df04391a79b8dc322cd67bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:38:20.590645 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1c2be2ae-b598-4168-b69a-4ed77401a368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:20.591280 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1c2be2ae-b598-4168-b69a-4ed77401a368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:20.592932 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1126/2271] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:38:20 2022] DELETE /identity/v3/projects/29ae56a95df04391a79b8dc322cd67bc => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:20.598670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ef8a9-cd78-44d7-ab06-fd9248485713 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:20.598982 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ef8a9-cd78-44d7-ab06-fd9248485713 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:20.599255 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ef8a9-cd78-44d7-ab06-fd9248485713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:20.620204 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-db2ef8a9-cd78-44d7-ab06-fd9248485713 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:20.621458 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-db2ef8a9-cd78-44d7-ab06-fd9248485713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:20.639838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1146/2272] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:38:20.691046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1127/2273] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:38:20.700326 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb1c9c-c6ec-4e71-9253-e4b2fa751773 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:20.700833 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb1c9c-c6ec-4e71-9253-e4b2fa751773 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:20.701047 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb1c9c-c6ec-4e71-9253-e4b2fa751773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:20.721660 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-9afb1c9c-c6ec-4e71-9253-e4b2fa751773 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:20.722923 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9afb1c9c-c6ec-4e71-9253-e4b2fa751773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:20.732089 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1147/2274] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:38:20.737009 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ef8a9-cd78-44d7-ab06-fd9248485713 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:20.748129 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ef8a9-cd78-44d7-ab06-fd9248485713 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:20.752845 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c491c4-305e-4992-985f-2d0194d18448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:20.753453 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c491c4-305e-4992-985f-2d0194d18448 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:20.753734 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c491c4-305e-4992-985f-2d0194d18448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:20.754008 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c491c4-305e-4992-985f-2d0194d18448 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:20.754652 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c491c4-305e-4992-985f-2d0194d18448 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:38:20.754954 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c491c4-305e-4992-985f-2d0194d18448 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:20.756840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c491c4-305e-4992-985f-2d0194d18448 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:20.769127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1128/2275] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:38:20.774151 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb1c9c-c6ec-4e71-9253-e4b2fa751773 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:20.783836 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb1c9c-c6ec-4e71-9253-e4b2fa751773 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:20.788531 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855251e0-7b55-437e-8bd5-8157a6368b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:20.789716 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855251e0-7b55-437e-8bd5-8157a6368b35 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:20.790310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855251e0-7b55-437e-8bd5-8157a6368b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:20.790578 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855251e0-7b55-437e-8bd5-8157a6368b35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:20.791209 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855251e0-7b55-437e-8bd5-8157a6368b35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:20.793127 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855251e0-7b55-437e-8bd5-8157a6368b35 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:20.821283 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1148/2276] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:38:20 2022] POST /identity/v3/projects => generated 409 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:20.825073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c491c4-305e-4992-985f-2d0194d18448 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:20.835297 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c491c4-305e-4992-985f-2d0194d18448 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:20.889085 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:20.899398 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb9de3-2d0d-4867-a62b-354672b71f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:20.900024 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb9de3-2d0d-4867-a62b-354672b71f92 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:20.901498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb9de3-2d0d-4867-a62b-354672b71f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:20.901498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb9de3-2d0d-4867-a62b-354672b71f92 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:20.902108 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb9de3-2d0d-4867-a62b-354672b71f92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:20.903714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb9de3-2d0d-4867-a62b-354672b71f92 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:20.931190 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1129/2277] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:38:20 2022] POST /identity/v3/users => generated 305 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:38:20.935405 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855251e0-7b55-437e-8bd5-8157a6368b35 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:20.946933 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855251e0-7b55-437e-8bd5-8157a6368b35 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:20.951610 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb401b-39d4-412a-8d9e-af0267540eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:20.952404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb401b-39d4-412a-8d9e-af0267540eb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:20.952969 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb401b-39d4-412a-8d9e-af0267540eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:20.953726 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb401b-39d4-412a-8d9e-af0267540eb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:20.954374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cb401b-39d4-412a-8d9e-af0267540eb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:20.955963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cb401b-39d4-412a-8d9e-af0267540eb3 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:20.969233 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1149/2278] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:38:20.971373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb9de3-2d0d-4867-a62b-354672b71f92 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:20.982771 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb9de3-2d0d-4867-a62b-354672b71f92 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:20.989083 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7cfd7-556e-4c31-8a38-8e867b746c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:20.989765 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7cfd7-556e-4c31-8a38-8e867b746c64 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:20.990046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7cfd7-556e-4c31-8a38-8e867b746c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:20.990354 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7cfd7-556e-4c31-8a38-8e867b746c64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:20.991002 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7cfd7-556e-4c31-8a38-8e867b746c64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:20.993162 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7cfd7-556e-4c31-8a38-8e867b746c64 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:21.005616 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1130/2279] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:38:21.010382 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb401b-39d4-412a-8d9e-af0267540eb3 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:21.020229 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb401b-39d4-412a-8d9e-af0267540eb3 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:21.024934 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908e0d4-4098-480f-b099-e17212d8217f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:21.025607 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908e0d4-4098-480f-b099-e17212d8217f admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:21.025882 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908e0d4-4098-480f-b099-e17212d8217f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:21.026148 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908e0d4-4098-480f-b099-e17212d8217f admin admin] PATH_INFO: `/v3/projects/ab07404138b140549ea4aadef04b82b2/users/cb70df703a924dcfa6e404086bee8b45/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:21.041026 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2908e0d4-4098-480f-b099-e17212d8217f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab07404138b140549ea4aadef04b82b2, user_id=cb70df703a924dcfa6e404086bee8b45, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:21.043652 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2908e0d4-4098-480f-b099-e17212d8217f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:21.057782 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1150/2280] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:38:21 2022] PUT /identity/v3/projects/ab07404138b140549ea4aadef04b82b2/users/cb70df703a924dcfa6e404086bee8b45/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:21.068764 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77a6e3-6333-4f8c-9b17-c3a5262481f3 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:21.069048 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77a6e3-6333-4f8c-9b17-c3a5262481f3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:21.069320 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77a6e3-6333-4f8c-9b17-c3a5262481f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:21.097338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-8e77a6e3-6333-4f8c-9b17-c3a5262481f3 None None] MFA Rules not processed for user `cb70df703a924dcfa6e404086bee8b45`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:21.098673 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8e77a6e3-6333-4f8c-9b17-c3a5262481f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:21.162747 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1131/2281] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:21 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:21.167913 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908e0d4-4098-480f-b099-e17212d8217f admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:21.228284 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:21.242570 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908e0d4-4098-480f-b099-e17212d8217f admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:21.247398 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec91a20-1cc9-48e0-91ee-fbd093c154cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:21.248020 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec91a20-1cc9-48e0-91ee-fbd093c154cb admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:21.248302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec91a20-1cc9-48e0-91ee-fbd093c154cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:21.248577 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec91a20-1cc9-48e0-91ee-fbd093c154cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:21.249249 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec91a20-1cc9-48e0-91ee-fbd093c154cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:21.251295 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec91a20-1cc9-48e0-91ee-fbd093c154cb admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:21.287053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1151/2282] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:38:21 2022] POST /identity/v3/projects => generated 409 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:21.287967 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77a6e3-6333-4f8c-9b17-c3a5262481f3 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:21.300746 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77a6e3-6333-4f8c-9b17-c3a5262481f3 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:21.366710 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe8dc8-d9c0-4fc8-85e6-0a2c758fa891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:21.368236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe8dc8-d9c0-4fc8-85e6-0a2c758fa891 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:21.368485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe8dc8-d9c0-4fc8-85e6-0a2c758fa891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:21.368795 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe8dc8-d9c0-4fc8-85e6-0a2c758fa891 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:21.369451 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fe8dc8-d9c0-4fc8-85e6-0a2c758fa891 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:21.371425 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fe8dc8-d9c0-4fc8-85e6-0a2c758fa891 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:21.394764 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1132/2283] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:38:21 2022] POST /identity/v3/users => generated 311 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:38:21.398629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec91a20-1cc9-48e0-91ee-fbd093c154cb admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:21.408941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec91a20-1cc9-48e0-91ee-fbd093c154cb admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:21.413463 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab61d4-5652-45a2-8fbe-db7ba0d3968b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:21.414569 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab61d4-5652-45a2-8fbe-db7ba0d3968b admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:21.414840 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab61d4-5652-45a2-8fbe-db7ba0d3968b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:21.415105 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab61d4-5652-45a2-8fbe-db7ba0d3968b admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:21.415737 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab61d4-5652-45a2-8fbe-db7ba0d3968b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:21.417336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab61d4-5652-45a2-8fbe-db7ba0d3968b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:21.429560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1152/2284] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:38:21.433941 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe8dc8-d9c0-4fc8-85e6-0a2c758fa891 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:21.444534 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe8dc8-d9c0-4fc8-85e6-0a2c758fa891 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:21.452334 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed966a-978e-4fbc-a620-629cc2226bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:21.453215 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed966a-978e-4fbc-a620-629cc2226bb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:21.453548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed966a-978e-4fbc-a620-629cc2226bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:21.453913 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed966a-978e-4fbc-a620-629cc2226bb2 admin admin] PATH_INFO: `/v3/projects/efd547ba5c4b4d3798ac9fc41aa3929c/users/73d5b02dd3b34bd69a4b5b74769717ed/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:21.473003 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed966a-978e-4fbc-a620-629cc2226bb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efd547ba5c4b4d3798ac9fc41aa3929c, user_id=73d5b02dd3b34bd69a4b5b74769717ed, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:21.475783 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed966a-978e-4fbc-a620-629cc2226bb2 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:21.498853 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1133/2285] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:38:21 2022] PUT /identity/v3/projects/efd547ba5c4b4d3798ac9fc41aa3929c/users/73d5b02dd3b34bd69a4b5b74769717ed/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:21.502816 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab61d4-5652-45a2-8fbe-db7ba0d3968b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:21.514053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab61d4-5652-45a2-8fbe-db7ba0d3968b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:21.595508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a221e84-3502-42e3-8c00-f17a324b2aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:21.596424 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a221e84-3502-42e3-8c00-f17a324b2aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:21.597002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a221e84-3502-42e3-8c00-f17a324b2aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:21.597338 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a221e84-3502-42e3-8c00-f17a324b2aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:21.598188 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a221e84-3502-42e3-8c00-f17a324b2aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:21.600061 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a221e84-3502-42e3-8c00-f17a324b2aa2 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:21.620729 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1153/2286] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:38:21.623641 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed966a-978e-4fbc-a620-629cc2226bb2 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:21.637891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed966a-978e-4fbc-a620-629cc2226bb2 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:21.643168 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367efd7-4876-47b3-bc75-e416a73559d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:21.644056 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367efd7-4876-47b3-bc75-e416a73559d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:21.644418 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367efd7-4876-47b3-bc75-e416a73559d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:21.644773 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367efd7-4876-47b3-bc75-e416a73559d5 admin admin] PATH_INFO: `/v3/projects/efd547ba5c4b4d3798ac9fc41aa3929c/users/73d5b02dd3b34bd69a4b5b74769717ed/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:21.646852 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367efd7-4876-47b3-bc75-e416a73559d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efd547ba5c4b4d3798ac9fc41aa3929c, user_id=73d5b02dd3b34bd69a4b5b74769717ed, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:21.649864 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367efd7-4876-47b3-bc75-e416a73559d5 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:21.661068 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1134/2287] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:38:21 2022] PUT /identity/v3/projects/efd547ba5c4b4d3798ac9fc41aa3929c/users/73d5b02dd3b34bd69a4b5b74769717ed/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:38:21.671471 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c204e8-e3d7-483e-8e99-1651d7717ccd None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:21.671471 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c204e8-e3d7-483e-8e99-1651d7717ccd None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:21.671733 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c204e8-e3d7-483e-8e99-1651d7717ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:21.705609 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-31c204e8-e3d7-483e-8e99-1651d7717ccd None None] MFA Rules not processed for user `73d5b02dd3b34bd69a4b5b74769717ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:21.706883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-31c204e8-e3d7-483e-8e99-1651d7717ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:21.792561 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1154/2288] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:21 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:38:21.803582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367efd7-4876-47b3-bc75-e416a73559d5 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:21.807359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3367efd7-4876-47b3-bc75-e416a73559d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:21.827882 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:21.869998 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367efd7-4876-47b3-bc75-e416a73559d5 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:21.873922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66260df8-a7e4-4a32-ba5c-b8da204ca26e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:21.874697 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66260df8-a7e4-4a32-ba5c-b8da204ca26e service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:21.874973 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66260df8-a7e4-4a32-ba5c-b8da204ca26e service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:21.875243 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66260df8-a7e4-4a32-ba5c-b8da204ca26e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:21.885421 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66260df8-a7e4-4a32-ba5c-b8da204ca26e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:21.886866 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66260df8-a7e4-4a32-ba5c-b8da204ca26e service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:21.896545 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1135/2289] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:21 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:38:22.073920 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c204e8-e3d7-483e-8e99-1651d7717ccd None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:22.083588 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c204e8-e3d7-483e-8e99-1651d7717ccd None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:22.087470 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bbcc9-0097-44e4-8c3e-e1beb268ed2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:22.088072 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bbcc9-0097-44e4-8c3e-e1beb268ed2e service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:22.088359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bbcc9-0097-44e4-8c3e-e1beb268ed2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:22.088616 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bbcc9-0097-44e4-8c3e-e1beb268ed2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:22.132938 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:22.145843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bbcc9-0097-44e4-8c3e-e1beb268ed2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:22.147262 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bbcc9-0097-44e4-8c3e-e1beb268ed2e service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:22.156160 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1155/2290] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:22 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:38:22.738370 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b0fa9-3c56-4a8e-9d96-4eb6e2579d0e None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:22.738639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b0fa9-3c56-4a8e-9d96-4eb6e2579d0e None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:22.738898 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b0fa9-3c56-4a8e-9d96-4eb6e2579d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:22.759964 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-750b0fa9-3c56-4a8e-9d96-4eb6e2579d0e None None] MFA Rules not processed for user `7a59f547a20e48ee88e3e620ba1e87f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:22.761229 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-750b0fa9-3c56-4a8e-9d96-4eb6e2579d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:22.820824 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1136/2291] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Oct 20 06:38:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:38:22.834325 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bbcc9-0097-44e4-8c3e-e1beb268ed2e service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:22.835839 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ab2bbcc9-0097-44e4-8c3e-e1beb268ed2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:22.921283 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bbcc9-0097-44e4-8c3e-e1beb268ed2e service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:22.927175 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe394273-d839-4b41-84e1-f47a04beb9b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeeab506ab5c4d6386fc899086f0abf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:22.927787 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe394273-d839-4b41-84e1-f47a04beb9b3 service swift] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:22.928060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe394273-d839-4b41-84e1-f47a04beb9b3 service swift] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:22.928981 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe394273-d839-4b41-84e1-f47a04beb9b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:22.938583 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe394273-d839-4b41-84e1-f47a04beb9b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:22.940338 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe394273-d839-4b41-84e1-f47a04beb9b3 service swift] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:22.950179 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1156/2292] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Oct 20 06:38:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 20 06:38:30.482143 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b0fa9-3c56-4a8e-9d96-4eb6e2579d0e None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:30.483836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-750b0fa9-3c56-4a8e-9d96-4eb6e2579d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:30.564054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b0fa9-3c56-4a8e-9d96-4eb6e2579d0e None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:30.583096 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf4cc4-376f-4ae4-be3e-f0511daba119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:30.583096 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf4cc4-376f-4ae4-be3e-f0511daba119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:30.583096 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf4cc4-376f-4ae4-be3e-f0511daba119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:30.583096 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf4cc4-376f-4ae4-be3e-f0511daba119 admin admin] PATH_INFO: `/v3/users/e1b86ae422b248829d1189c8013c733a` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:30.583786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf4cc4-376f-4ae4-be3e-f0511daba119 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1b86ae422b248829d1189c8013c733a)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:30.583786 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf4cc4-376f-4ae4-be3e-f0511daba119 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:30.631972 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-78bf4cc4-376f-4ae4-be3e-f0511daba119 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1b86ae422b248829d1189c8013c733a'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:30.632654 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-78bf4cc4-376f-4ae4-be3e-f0511daba119 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1b86ae422b248829d1189c8013c733a'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:30.640823 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-78bf4cc4-376f-4ae4-be3e-f0511daba119 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1b86ae422b248829d1189c8013c733a'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:30.648674 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:30.655105 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-78bf4cc4-376f-4ae4-be3e-f0511daba119 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1b86ae422b248829d1189c8013c733a'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:30.662681 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-78bf4cc4-376f-4ae4-be3e-f0511daba119 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1b86ae422b248829d1189c8013c733a'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:30.669173 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1137/2293] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:38:30 2022] DELETE /identity/v3/users/e1b86ae422b248829d1189c8013c733a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:30.673656 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba040983-45eb-4d02-ac42-1f8a3b3af188 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:30.674173 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba040983-45eb-4d02-ac42-1f8a3b3af188 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:30.674173 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba040983-45eb-4d02-ac42-1f8a3b3af188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:30.696252 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-ba040983-45eb-4d02-ac42-1f8a3b3af188 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:30.697629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ba040983-45eb-4d02-ac42-1f8a3b3af188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:30.780315 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1157/2294] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:38:30.790517 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf4cc4-376f-4ae4-be3e-f0511daba119 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:30.791950 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-78bf4cc4-376f-4ae4-be3e-f0511daba119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:30.867863 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf4cc4-376f-4ae4-be3e-f0511daba119 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:30.871744 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581ecd5-c3be-4f92-a23e-bdf03c54335d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:30.872348 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581ecd5-c3be-4f92-a23e-bdf03c54335d service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:30.872622 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581ecd5-c3be-4f92-a23e-bdf03c54335d service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:30.872923 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581ecd5-c3be-4f92-a23e-bdf03c54335d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:30.883233 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581ecd5-c3be-4f92-a23e-bdf03c54335d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:30.884747 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581ecd5-c3be-4f92-a23e-bdf03c54335d service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:30.895637 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1138/2295] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:38:31.302771 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba040983-45eb-4d02-ac42-1f8a3b3af188 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:31.304355 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ba040983-45eb-4d02-ac42-1f8a3b3af188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:31.321741 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba040983-45eb-4d02-ac42-1f8a3b3af188 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:31.327525 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae557e-f40b-4428-93b6-9947495bcf6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:31.328290 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae557e-f40b-4428-93b6-9947495bcf6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:31.328574 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae557e-f40b-4428-93b6-9947495bcf6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:31.328895 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae557e-f40b-4428-93b6-9947495bcf6e admin admin] PATH_INFO: `/v3/projects/50afaa2b012f43de81324f9f94d95803` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:31.330107 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae557e-f40b-4428-93b6-9947495bcf6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=50afaa2b012f43de81324f9f94d95803)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:31.332062 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae557e-f40b-4428-93b6-9947495bcf6e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:31.378170 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-1fae557e-f40b-4428-93b6-9947495bcf6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50afaa2b012f43de81324f9f94d95803', 'request_id': 'req-1fae557e-f40b-4428-93b6-9947495bcf6e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:31.384432 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-1fae557e-f40b-4428-93b6-9947495bcf6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50afaa2b012f43de81324f9f94d95803', 'request_id': 'req-1fae557e-f40b-4428-93b6-9947495bcf6e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:31.392249 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-1fae557e-f40b-4428-93b6-9947495bcf6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50afaa2b012f43de81324f9f94d95803', 'request_id': 'req-1fae557e-f40b-4428-93b6-9947495bcf6e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:31.422711 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:31.430193 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-1fae557e-f40b-4428-93b6-9947495bcf6e admin admin] The token cache is being invalidate because project 50afaa2b012f43de81324f9f94d95803 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:38:31.430681 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-1fae557e-f40b-4428-93b6-9947495bcf6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:31.432236 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-1fae557e-f40b-4428-93b6-9947495bcf6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:31.437074 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1158/2296] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:38:31 2022] DELETE /identity/v3/projects/50afaa2b012f43de81324f9f94d95803 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:31.438399 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581ecd5-c3be-4f92-a23e-bdf03c54335d service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:31.448264 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581ecd5-c3be-4f92-a23e-bdf03c54335d service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:31.524424 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50965f3d-a32e-4b4b-86d7-c4b14a03c23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:31.525168 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50965f3d-a32e-4b4b-86d7-c4b14a03c23e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:31.525363 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50965f3d-a32e-4b4b-86d7-c4b14a03c23e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:31.525640 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50965f3d-a32e-4b4b-86d7-c4b14a03c23e admin admin] PATH_INFO: `/v3/users/9c568917c053466684d64ad853fc08ff` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:31.526779 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50965f3d-a32e-4b4b-86d7-c4b14a03c23e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c568917c053466684d64ad853fc08ff)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:31.528794 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50965f3d-a32e-4b4b-86d7-c4b14a03c23e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:31.578115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-50965f3d-a32e-4b4b-86d7-c4b14a03c23e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c568917c053466684d64ad853fc08ff'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:31.579520 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-50965f3d-a32e-4b4b-86d7-c4b14a03c23e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c568917c053466684d64ad853fc08ff'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:31.593387 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-50965f3d-a32e-4b4b-86d7-c4b14a03c23e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c568917c053466684d64ad853fc08ff'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:31.635213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-50965f3d-a32e-4b4b-86d7-c4b14a03c23e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c568917c053466684d64ad853fc08ff'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:31.656358 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-50965f3d-a32e-4b4b-86d7-c4b14a03c23e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c568917c053466684d64ad853fc08ff'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:31.685682 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1139/2297] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:38:31 2022] DELETE /identity/v3/users/9c568917c053466684d64ad853fc08ff => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:32.048576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae557e-f40b-4428-93b6-9947495bcf6e admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:32.050191 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1fae557e-f40b-4428-93b6-9947495bcf6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:32.085195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae557e-f40b-4428-93b6-9947495bcf6e admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:32.159035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e835de8-b6fa-4081-9a00-c2738927b2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:32.159762 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e835de8-b6fa-4081-9a00-c2738927b2bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:32.160112 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e835de8-b6fa-4081-9a00-c2738927b2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:32.160471 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e835de8-b6fa-4081-9a00-c2738927b2bd admin admin] PATH_INFO: `/v3/projects/87df72e1192a4578b770785dc5ed12fc` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:32.161708 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e835de8-b6fa-4081-9a00-c2738927b2bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=87df72e1192a4578b770785dc5ed12fc)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:32.163611 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e835de8-b6fa-4081-9a00-c2738927b2bd admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:32.268826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-4e835de8-b6fa-4081-9a00-c2738927b2bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87df72e1192a4578b770785dc5ed12fc', 'request_id': 'req-4e835de8-b6fa-4081-9a00-c2738927b2bd'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:32.287377 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-4e835de8-b6fa-4081-9a00-c2738927b2bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87df72e1192a4578b770785dc5ed12fc', 'request_id': 'req-4e835de8-b6fa-4081-9a00-c2738927b2bd'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:32.312908 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-4e835de8-b6fa-4081-9a00-c2738927b2bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87df72e1192a4578b770785dc5ed12fc', 'request_id': 'req-4e835de8-b6fa-4081-9a00-c2738927b2bd'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:32.336061 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-4e835de8-b6fa-4081-9a00-c2738927b2bd admin admin] The token cache is being invalidate because project 87df72e1192a4578b770785dc5ed12fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:38:32.337051 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-4e835de8-b6fa-4081-9a00-c2738927b2bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:32.338449 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-4e835de8-b6fa-4081-9a00-c2738927b2bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:32.341651 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1159/2298] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:38:32 2022] DELETE /identity/v3/projects/87df72e1192a4578b770785dc5ed12fc => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:32.348163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd62b01-387b-47ed-bf49-144ffdbfc4b6 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:32.348867 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd62b01-387b-47ed-bf49-144ffdbfc4b6 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:32.349341 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd62b01-387b-47ed-bf49-144ffdbfc4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:32.362389 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:32.384731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-7dd62b01-387b-47ed-bf49-144ffdbfc4b6 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:32.384731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7dd62b01-387b-47ed-bf49-144ffdbfc4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:32.460659 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1140/2299] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:32.470965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e0ee4-d268-4ecf-8e7e-a677625f8aa3 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:32.471348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e0ee4-d268-4ecf-8e7e-a677625f8aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:32.471729 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e0ee4-d268-4ecf-8e7e-a677625f8aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:32.492982 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-fe3e0ee4-d268-4ecf-8e7e-a677625f8aa3 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:32.494192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-fe3e0ee4-d268-4ecf-8e7e-a677625f8aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:32.503051 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1160/2300] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:32.507547 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd62b01-387b-47ed-bf49-144ffdbfc4b6 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:32.519164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd62b01-387b-47ed-bf49-144ffdbfc4b6 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:32.523944 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65300c6e-6887-40a8-950a-5accb069ee44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:32.524555 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65300c6e-6887-40a8-950a-5accb069ee44 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:32.525174 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65300c6e-6887-40a8-950a-5accb069ee44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:32.525174 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65300c6e-6887-40a8-950a-5accb069ee44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:32.525766 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65300c6e-6887-40a8-950a-5accb069ee44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:38:32.526092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65300c6e-6887-40a8-950a-5accb069ee44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:32.528426 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65300c6e-6887-40a8-950a-5accb069ee44 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:32.540572 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1141/2301] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:38:32.545105 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e0ee4-d268-4ecf-8e7e-a677625f8aa3 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:32.554218 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e0ee4-d268-4ecf-8e7e-a677625f8aa3 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:32.559189 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e152c4c-d2ca-4bd2-af84-512f67007504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:32.559582 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e152c4c-d2ca-4bd2-af84-512f67007504 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:32.559855 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e152c4c-d2ca-4bd2-af84-512f67007504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:32.560117 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e152c4c-d2ca-4bd2-af84-512f67007504 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:32.560781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e152c4c-d2ca-4bd2-af84-512f67007504 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:32.562719 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e152c4c-d2ca-4bd2-af84-512f67007504 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:32.609490 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1161/2302] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:38:32 2022] POST /identity/v3/projects => generated 381 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:32.611039 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65300c6e-6887-40a8-950a-5accb069ee44 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:32.624898 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65300c6e-6887-40a8-950a-5accb069ee44 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:32.706387 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8ddf8-cb0a-41fc-ab9e-b681db7e138d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:32.707019 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8ddf8-cb0a-41fc-ab9e-b681db7e138d admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:32.707299 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8ddf8-cb0a-41fc-ab9e-b681db7e138d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:32.707606 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8ddf8-cb0a-41fc-ab9e-b681db7e138d admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:32.708238 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8ddf8-cb0a-41fc-ab9e-b681db7e138d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:32.710187 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8ddf8-cb0a-41fc-ab9e-b681db7e138d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:32.734594 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1142/2303] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:38:32 2022] POST /identity/v3/users => generated 291 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:38:32.738658 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e152c4c-d2ca-4bd2-af84-512f67007504 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:32.749175 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e152c4c-d2ca-4bd2-af84-512f67007504 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:32.753782 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be151555-ea7e-47c2-876d-173a2c187399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:32.754378 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be151555-ea7e-47c2-876d-173a2c187399 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:32.754638 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be151555-ea7e-47c2-876d-173a2c187399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:32.754904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be151555-ea7e-47c2-876d-173a2c187399 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:32.755782 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be151555-ea7e-47c2-876d-173a2c187399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:32.757382 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be151555-ea7e-47c2-876d-173a2c187399 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:32.767275 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1162/2304] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:38:32.771085 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8ddf8-cb0a-41fc-ab9e-b681db7e138d admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:32.781311 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8ddf8-cb0a-41fc-ab9e-b681db7e138d admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:32.785922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381a9e8-b3c5-4524-9212-5c672197e74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:32.786510 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381a9e8-b3c5-4524-9212-5c672197e74b admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:32.786785 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381a9e8-b3c5-4524-9212-5c672197e74b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:32.787051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381a9e8-b3c5-4524-9212-5c672197e74b admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:32.787670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381a9e8-b3c5-4524-9212-5c672197e74b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:32.789301 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381a9e8-b3c5-4524-9212-5c672197e74b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:32.800556 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1143/2305] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:38:32.812019 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be151555-ea7e-47c2-876d-173a2c187399 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:32.830249 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be151555-ea7e-47c2-876d-173a2c187399 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:32.837263 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17ffeb-9808-4adf-8fd4-55e3333b703a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:32.837916 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17ffeb-9808-4adf-8fd4-55e3333b703a admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:32.838196 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17ffeb-9808-4adf-8fd4-55e3333b703a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:32.838471 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17ffeb-9808-4adf-8fd4-55e3333b703a admin admin] PATH_INFO: `/v3/projects/dce0c87a532c497b97e06b7f22d72ebb/users/84ec0b089da74156aaec034dbbac0895/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:32.856930 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c17ffeb-9808-4adf-8fd4-55e3333b703a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce0c87a532c497b97e06b7f22d72ebb, user_id=84ec0b089da74156aaec034dbbac0895, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:32.859589 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c17ffeb-9808-4adf-8fd4-55e3333b703a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:32.889343 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1163/2306] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:38:32 2022] PUT /identity/v3/projects/dce0c87a532c497b97e06b7f22d72ebb/users/84ec0b089da74156aaec034dbbac0895/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:32.891874 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de284206-a343-43db-9648-6ff3b4bdd385 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:32.892113 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de284206-a343-43db-9648-6ff3b4bdd385 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:32.892390 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de284206-a343-43db-9648-6ff3b4bdd385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:32.946021 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-de284206-a343-43db-9648-6ff3b4bdd385 None None] MFA Rules not processed for user `84ec0b089da74156aaec034dbbac0895`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:32.947261 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-de284206-a343-43db-9648-6ff3b4bdd385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:33.028499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:33.042232 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1144/2307] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:33.054568 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17ffeb-9808-4adf-8fd4-55e3333b703a admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:33.057014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5c17ffeb-9808-4adf-8fd4-55e3333b703a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:33.096621 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de284206-a343-43db-9648-6ff3b4bdd385 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:33.098394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-de284206-a343-43db-9648-6ff3b4bdd385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:33.130732 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:33.179796 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17ffeb-9808-4adf-8fd4-55e3333b703a admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:33.183597 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39519609-7e4c-4179-bf8a-c9766acad563 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:33.184205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39519609-7e4c-4179-bf8a-c9766acad563 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:33.184503 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39519609-7e4c-4179-bf8a-c9766acad563 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:33.184797 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39519609-7e4c-4179-bf8a-c9766acad563 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:33.193205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de284206-a343-43db-9648-6ff3b4bdd385 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:33.193481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39519609-7e4c-4179-bf8a-c9766acad563 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:33.194902 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39519609-7e4c-4179-bf8a-c9766acad563 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:33.197701 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd952c-e7a5-4232-a28b-e1c58d138d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:33.198375 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd952c-e7a5-4232-a28b-e1c58d138d3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:33.199302 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd952c-e7a5-4232-a28b-e1c58d138d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:33.199614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd952c-e7a5-4232-a28b-e1c58d138d3e admin admin] PATH_INFO: `/v3/users/cb70df703a924dcfa6e404086bee8b45` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:33.200755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd952c-e7a5-4232-a28b-e1c58d138d3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb70df703a924dcfa6e404086bee8b45)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:33.202660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd952c-e7a5-4232-a28b-e1c58d138d3e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:33.204460 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1164/2308] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:38:33.251952 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a8dd952c-e7a5-4232-a28b-e1c58d138d3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb70df703a924dcfa6e404086bee8b45'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:33.252878 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a8dd952c-e7a5-4232-a28b-e1c58d138d3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb70df703a924dcfa6e404086bee8b45'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:33.261682 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a8dd952c-e7a5-4232-a28b-e1c58d138d3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb70df703a924dcfa6e404086bee8b45'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:33.274264 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a8dd952c-e7a5-4232-a28b-e1c58d138d3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb70df703a924dcfa6e404086bee8b45'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:33.281271 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a8dd952c-e7a5-4232-a28b-e1c58d138d3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb70df703a924dcfa6e404086bee8b45'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:33.287073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1145/2309] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:38:33 2022] DELETE /identity/v3/users/cb70df703a924dcfa6e404086bee8b45 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:33.291766 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6d198-748f-4542-b755-c35314a27601 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:33.292054 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6d198-748f-4542-b755-c35314a27601 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:33.292858 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6d198-748f-4542-b755-c35314a27601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:33.314096 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-31e6d198-748f-4542-b755-c35314a27601 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:33.315302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-31e6d198-748f-4542-b755-c35314a27601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:33.399798 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1165/2310] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:33.413340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd952c-e7a5-4232-a28b-e1c58d138d3e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:33.419544 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a8dd952c-e7a5-4232-a28b-e1c58d138d3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:33.497620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd952c-e7a5-4232-a28b-e1c58d138d3e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:33.502143 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd36a42-6d9b-4a7a-af7d-689109bcbe3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:33.503218 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd36a42-6d9b-4a7a-af7d-689109bcbe3c service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:33.503218 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd36a42-6d9b-4a7a-af7d-689109bcbe3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:33.503570 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd36a42-6d9b-4a7a-af7d-689109bcbe3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:33.517451 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd36a42-6d9b-4a7a-af7d-689109bcbe3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:33.519029 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd36a42-6d9b-4a7a-af7d-689109bcbe3c service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:33.531892 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1146/2311] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:38:33.921648 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6d198-748f-4542-b755-c35314a27601 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:33.923055 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-31e6d198-748f-4542-b755-c35314a27601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:33.937549 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6d198-748f-4542-b755-c35314a27601 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:33.942201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3492f7-215c-47b5-8700-d7cd325951c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:33.942841 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3492f7-215c-47b5-8700-d7cd325951c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:33.943123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3492f7-215c-47b5-8700-d7cd325951c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:33.943398 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3492f7-215c-47b5-8700-d7cd325951c8 admin admin] PATH_INFO: `/v3/projects/ab07404138b140549ea4aadef04b82b2` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:33.944519 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3492f7-215c-47b5-8700-d7cd325951c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab07404138b140549ea4aadef04b82b2)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:33.946446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3492f7-215c-47b5-8700-d7cd325951c8 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:33.994175 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-7c3492f7-215c-47b5-8700-d7cd325951c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab07404138b140549ea4aadef04b82b2', 'request_id': 'req-7c3492f7-215c-47b5-8700-d7cd325951c8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:34.000050 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-7c3492f7-215c-47b5-8700-d7cd325951c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab07404138b140549ea4aadef04b82b2', 'request_id': 'req-7c3492f7-215c-47b5-8700-d7cd325951c8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:34.007195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-7c3492f7-215c-47b5-8700-d7cd325951c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab07404138b140549ea4aadef04b82b2', 'request_id': 'req-7c3492f7-215c-47b5-8700-d7cd325951c8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:34.034860 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-7c3492f7-215c-47b5-8700-d7cd325951c8 admin admin] The token cache is being invalidate because project ab07404138b140549ea4aadef04b82b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:38:34.035338 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-7c3492f7-215c-47b5-8700-d7cd325951c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:34.036246 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-7c3492f7-215c-47b5-8700-d7cd325951c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:34.038349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1166/2312] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:38:33 2022] DELETE /identity/v3/projects/ab07404138b140549ea4aadef04b82b2 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:34.042646 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd36a42-6d9b-4a7a-af7d-689109bcbe3c service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:34.058489 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd36a42-6d9b-4a7a-af7d-689109bcbe3c service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:34.081260 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:34.124019 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f3e50-5765-40a5-a640-9889ae732e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:34.124648 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f3e50-5765-40a5-a640-9889ae732e60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:34.124954 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f3e50-5765-40a5-a640-9889ae732e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:34.125225 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f3e50-5765-40a5-a640-9889ae732e60 admin admin] PATH_INFO: `/v3/users/73d5b02dd3b34bd69a4b5b74769717ed` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:34.126332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f3e50-5765-40a5-a640-9889ae732e60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73d5b02dd3b34bd69a4b5b74769717ed)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:34.128238 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f3e50-5765-40a5-a640-9889ae732e60 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:34.174140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d27f3e50-5765-40a5-a640-9889ae732e60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73d5b02dd3b34bd69a4b5b74769717ed'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:34.174773 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d27f3e50-5765-40a5-a640-9889ae732e60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73d5b02dd3b34bd69a4b5b74769717ed'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:34.181850 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d27f3e50-5765-40a5-a640-9889ae732e60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73d5b02dd3b34bd69a4b5b74769717ed'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:34.195156 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d27f3e50-5765-40a5-a640-9889ae732e60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73d5b02dd3b34bd69a4b5b74769717ed'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:34.203048 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d27f3e50-5765-40a5-a640-9889ae732e60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73d5b02dd3b34bd69a4b5b74769717ed'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:34.209029 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1147/2313] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:38:34 2022] DELETE /identity/v3/users/73d5b02dd3b34bd69a4b5b74769717ed => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:34.483721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3492f7-215c-47b5-8700-d7cd325951c8 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:34.485287 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7c3492f7-215c-47b5-8700-d7cd325951c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:34.502634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3492f7-215c-47b5-8700-d7cd325951c8 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:34.554137 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:34.565748 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec89617-fe38-4574-ad66-3587687f1b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:34.566381 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec89617-fe38-4574-ad66-3587687f1b1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:34.566661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec89617-fe38-4574-ad66-3587687f1b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:34.566934 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec89617-fe38-4574-ad66-3587687f1b1f admin admin] PATH_INFO: `/v3/projects/efd547ba5c4b4d3798ac9fc41aa3929c` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:34.568058 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec89617-fe38-4574-ad66-3587687f1b1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=efd547ba5c4b4d3798ac9fc41aa3929c)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:34.569993 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec89617-fe38-4574-ad66-3587687f1b1f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:34.615655 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-0ec89617-fe38-4574-ad66-3587687f1b1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efd547ba5c4b4d3798ac9fc41aa3929c', 'request_id': 'req-0ec89617-fe38-4574-ad66-3587687f1b1f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:34.621917 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-0ec89617-fe38-4574-ad66-3587687f1b1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efd547ba5c4b4d3798ac9fc41aa3929c', 'request_id': 'req-0ec89617-fe38-4574-ad66-3587687f1b1f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:34.628244 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-0ec89617-fe38-4574-ad66-3587687f1b1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efd547ba5c4b4d3798ac9fc41aa3929c', 'request_id': 'req-0ec89617-fe38-4574-ad66-3587687f1b1f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:34.641179 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-0ec89617-fe38-4574-ad66-3587687f1b1f admin admin] The token cache is being invalidate because project efd547ba5c4b4d3798ac9fc41aa3929c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:38:34.641516 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-0ec89617-fe38-4574-ad66-3587687f1b1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:34.642196 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-0ec89617-fe38-4574-ad66-3587687f1b1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:34.643976 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1167/2314] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:38:34 2022] DELETE /identity/v3/projects/efd547ba5c4b4d3798ac9fc41aa3929c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:38:34.648851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ba046-3336-464b-8cef-cd5f6dd0690a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:34.649340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ba046-3336-464b-8cef-cd5f6dd0690a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:34.649442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ba046-3336-464b-8cef-cd5f6dd0690a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:34.670468 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-978ba046-3336-464b-8cef-cd5f6dd0690a None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:34.671680 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-978ba046-3336-464b-8cef-cd5f6dd0690a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:34.742672 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1148/2315] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:34.752078 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452601a7-2bf4-4e1f-a581-380c7dab950c None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:34.752449 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452601a7-2bf4-4e1f-a581-380c7dab950c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:34.752836 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452601a7-2bf4-4e1f-a581-380c7dab950c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:34.774496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-452601a7-2bf4-4e1f-a581-380c7dab950c None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:34.775805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-452601a7-2bf4-4e1f-a581-380c7dab950c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:34.784775 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1168/2316] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:34.788920 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ba046-3336-464b-8cef-cd5f6dd0690a None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:34.800472 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ba046-3336-464b-8cef-cd5f6dd0690a None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:34.805557 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da9da0-c906-413f-846d-7d41cfd6dfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:34.806392 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da9da0-c906-413f-846d-7d41cfd6dfae admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:34.806772 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da9da0-c906-413f-846d-7d41cfd6dfae admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:34.807124 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da9da0-c906-413f-846d-7d41cfd6dfae admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:34.807775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da9da0-c906-413f-846d-7d41cfd6dfae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:38:34.808199 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da9da0-c906-413f-846d-7d41cfd6dfae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:34.810278 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da9da0-c906-413f-846d-7d41cfd6dfae admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:34.827142 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1149/2317] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:38:34.828301 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452601a7-2bf4-4e1f-a581-380c7dab950c None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:34.842208 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452601a7-2bf4-4e1f-a581-380c7dab950c None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:34.847918 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34a89c-7299-4eee-8895-edf584ab39b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:34.848631 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34a89c-7299-4eee-8895-edf584ab39b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:34.849020 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34a89c-7299-4eee-8895-edf584ab39b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:34.849369 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34a89c-7299-4eee-8895-edf584ab39b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:34.850077 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34a89c-7299-4eee-8895-edf584ab39b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:34.852095 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34a89c-7299-4eee-8895-edf584ab39b4 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:34.884433 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1169/2318] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:38:34 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:34.890030 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da9da0-c906-413f-846d-7d41cfd6dfae admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:34.902737 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da9da0-c906-413f-846d-7d41cfd6dfae admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:34.954077 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:34.987851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d9e29-395f-416e-a0cd-0162538cfba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:34.988460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d9e29-395f-416e-a0cd-0162538cfba8 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:34.988765 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d9e29-395f-416e-a0cd-0162538cfba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:34.989035 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d9e29-395f-416e-a0cd-0162538cfba8 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:34.989663 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d9e29-395f-416e-a0cd-0162538cfba8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:34.991580 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d9e29-395f-416e-a0cd-0162538cfba8 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:35.016855 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1150/2319] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:38:34 2022] POST /identity/v3/users => generated 285 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:38:35.017998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34a89c-7299-4eee-8895-edf584ab39b4 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:35.029008 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34a89c-7299-4eee-8895-edf584ab39b4 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:35.033506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf97d1-5ad1-49e5-8395-98c3aa486b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:35.034092 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf97d1-5ad1-49e5-8395-98c3aa486b38 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:35.034366 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf97d1-5ad1-49e5-8395-98c3aa486b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:35.034635 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf97d1-5ad1-49e5-8395-98c3aa486b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:35.035258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbf97d1-5ad1-49e5-8395-98c3aa486b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:35.036872 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbf97d1-5ad1-49e5-8395-98c3aa486b38 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:35.047516 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1170/2320] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:38:35.051415 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d9e29-395f-416e-a0cd-0162538cfba8 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:35.061359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d9e29-395f-416e-a0cd-0162538cfba8 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:35.065958 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb131f75-7306-45b5-b37d-ee0be838a6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:35.066545 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb131f75-7306-45b5-b37d-ee0be838a6b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:35.066826 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb131f75-7306-45b5-b37d-ee0be838a6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:35.067100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb131f75-7306-45b5-b37d-ee0be838a6b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:35.067735 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb131f75-7306-45b5-b37d-ee0be838a6b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:35.069365 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb131f75-7306-45b5-b37d-ee0be838a6b0 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:35.080114 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1151/2321] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:38:35.084135 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf97d1-5ad1-49e5-8395-98c3aa486b38 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:35.094417 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf97d1-5ad1-49e5-8395-98c3aa486b38 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:35.099071 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d57852-5869-47f0-a440-c254436f9d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:35.099741 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d57852-5869-47f0-a440-c254436f9d7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:35.100022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d57852-5869-47f0-a440-c254436f9d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:35.100315 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d57852-5869-47f0-a440-c254436f9d7b admin admin] PATH_INFO: `/v3/projects/60b6684006b54f899e9a61f2d4db1a3e/users/a6dfca9a0b314fea96df45d7b9cf8e2b/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:35.115665 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d57852-5869-47f0-a440-c254436f9d7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b6684006b54f899e9a61f2d4db1a3e, user_id=a6dfca9a0b314fea96df45d7b9cf8e2b, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:35.118334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d57852-5869-47f0-a440-c254436f9d7b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:35.130750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1171/2322] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:38:35 2022] PUT /identity/v3/projects/60b6684006b54f899e9a61f2d4db1a3e/users/a6dfca9a0b314fea96df45d7b9cf8e2b/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:35.140977 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d797d-5fa9-4f31-89aa-adc3228f4f42 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:35.141208 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d797d-5fa9-4f31-89aa-adc3228f4f42 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:35.141486 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d797d-5fa9-4f31-89aa-adc3228f4f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:35.169404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-1d5d797d-5fa9-4f31-89aa-adc3228f4f42 None None] MFA Rules not processed for user `a6dfca9a0b314fea96df45d7b9cf8e2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:35.170602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1d5d797d-5fa9-4f31-89aa-adc3228f4f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:35.233972 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1152/2323] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:35.245983 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d57852-5869-47f0-a440-c254436f9d7b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:35.247680 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-37d57852-5869-47f0-a440-c254436f9d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:35.304150 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d57852-5869-47f0-a440-c254436f9d7b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:35.308580 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1be1e-e588-4428-b6ba-8c9243d7949c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:35.309387 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1be1e-e588-4428-b6ba-8c9243d7949c service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:35.309672 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1be1e-e588-4428-b6ba-8c9243d7949c service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:35.309947 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1be1e-e588-4428-b6ba-8c9243d7949c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:35.318522 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1be1e-e588-4428-b6ba-8c9243d7949c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:35.319951 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1be1e-e588-4428-b6ba-8c9243d7949c service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:35.328859 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1172/2324] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:35 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:38:35.353711 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d797d-5fa9-4f31-89aa-adc3228f4f42 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:35.415433 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d797d-5fa9-4f31-89aa-adc3228f4f42 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:35.420498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8736d-4902-433c-97d5-77e3b8f7054c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:35.421145 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8736d-4902-433c-97d5-77e3b8f7054c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:35.421424 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8736d-4902-433c-97d5-77e3b8f7054c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:35.421693 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8736d-4902-433c-97d5-77e3b8f7054c admin admin] PATH_INFO: `/v3/users/a6dfca9a0b314fea96df45d7b9cf8e2b` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:35.422803 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8736d-4902-433c-97d5-77e3b8f7054c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6dfca9a0b314fea96df45d7b9cf8e2b)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:35.425030 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8736d-4902-433c-97d5-77e3b8f7054c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:35.466118 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:35.475143 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6dc8736d-4902-433c-97d5-77e3b8f7054c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6dfca9a0b314fea96df45d7b9cf8e2b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:35.475880 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6dc8736d-4902-433c-97d5-77e3b8f7054c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6dfca9a0b314fea96df45d7b9cf8e2b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:35.485363 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6dc8736d-4902-433c-97d5-77e3b8f7054c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6dfca9a0b314fea96df45d7b9cf8e2b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:35.498189 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6dc8736d-4902-433c-97d5-77e3b8f7054c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6dfca9a0b314fea96df45d7b9cf8e2b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:35.505797 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6dc8736d-4902-433c-97d5-77e3b8f7054c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6dfca9a0b314fea96df45d7b9cf8e2b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:35.511807 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1153/2325] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:38:35 2022] DELETE /identity/v3/users/a6dfca9a0b314fea96df45d7b9cf8e2b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:35.516178 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780cd81-da8f-4fb1-b756-51601f7e4b9e None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:35.516629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780cd81-da8f-4fb1-b756-51601f7e4b9e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:35.516782 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780cd81-da8f-4fb1-b756-51601f7e4b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:35.536135 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a780cd81-da8f-4fb1-b756-51601f7e4b9e None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:35.537323 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a780cd81-da8f-4fb1-b756-51601f7e4b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:35.549949 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:35.597829 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1173/2326] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:35.607973 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8736d-4902-433c-97d5-77e3b8f7054c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:35.609385 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6dc8736d-4902-433c-97d5-77e3b8f7054c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:35.670695 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8736d-4902-433c-97d5-77e3b8f7054c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:35.674342 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9666f3c-6e09-4074-ac54-154c22efb3a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:35.674971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9666f3c-6e09-4074-ac54-154c22efb3a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:35.675248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9666f3c-6e09-4074-ac54-154c22efb3a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:35.675522 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9666f3c-6e09-4074-ac54-154c22efb3a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:35.685336 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9666f3c-6e09-4074-ac54-154c22efb3a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:35.686762 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9666f3c-6e09-4074-ac54-154c22efb3a9 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:35.696192 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1154/2327] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:38:36.049359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780cd81-da8f-4fb1-b756-51601f7e4b9e None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:36.049359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a780cd81-da8f-4fb1-b756-51601f7e4b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:36.055434 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780cd81-da8f-4fb1-b756-51601f7e4b9e None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:36.060236 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f7e10-a82a-4ba8-b92f-c35d3a3abe8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:36.060901 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f7e10-a82a-4ba8-b92f-c35d3a3abe8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:36.061328 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f7e10-a82a-4ba8-b92f-c35d3a3abe8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:36.061608 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f7e10-a82a-4ba8-b92f-c35d3a3abe8c admin admin] PATH_INFO: `/v3/projects/60b6684006b54f899e9a61f2d4db1a3e` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:36.062735 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f7e10-a82a-4ba8-b92f-c35d3a3abe8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b6684006b54f899e9a61f2d4db1a3e)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:36.064646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f7e10-a82a-4ba8-b92f-c35d3a3abe8c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:36.111706 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ca0f7e10-a82a-4ba8-b92f-c35d3a3abe8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b6684006b54f899e9a61f2d4db1a3e', 'request_id': 'req-ca0f7e10-a82a-4ba8-b92f-c35d3a3abe8c'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:36.117234 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ca0f7e10-a82a-4ba8-b92f-c35d3a3abe8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b6684006b54f899e9a61f2d4db1a3e', 'request_id': 'req-ca0f7e10-a82a-4ba8-b92f-c35d3a3abe8c'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:36.123392 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ca0f7e10-a82a-4ba8-b92f-c35d3a3abe8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b6684006b54f899e9a61f2d4db1a3e', 'request_id': 'req-ca0f7e10-a82a-4ba8-b92f-c35d3a3abe8c'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:36.135194 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ca0f7e10-a82a-4ba8-b92f-c35d3a3abe8c admin admin] The token cache is being invalidate because project 60b6684006b54f899e9a61f2d4db1a3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:38:36.135481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ca0f7e10-a82a-4ba8-b92f-c35d3a3abe8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:36.136120 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ca0f7e10-a82a-4ba8-b92f-c35d3a3abe8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:36.138139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1174/2328] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:38:36 2022] DELETE /identity/v3/projects/60b6684006b54f899e9a61f2d4db1a3e => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:42.879272 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9666f3c-6e09-4074-ac54-154c22efb3a9 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:42.881127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d9666f3c-6e09-4074-ac54-154c22efb3a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:42.894884 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9666f3c-6e09-4074-ac54-154c22efb3a9 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:42.954508 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db995152-51a0-46c5-9cad-5bf44c8c46f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:42.955143 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db995152-51a0-46c5-9cad-5bf44c8c46f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:42.955422 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db995152-51a0-46c5-9cad-5bf44c8c46f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:42.955696 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db995152-51a0-46c5-9cad-5bf44c8c46f2 admin admin] PATH_INFO: `/v3/users/84ec0b089da74156aaec034dbbac0895` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:42.956875 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db995152-51a0-46c5-9cad-5bf44c8c46f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84ec0b089da74156aaec034dbbac0895)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:42.958791 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db995152-51a0-46c5-9cad-5bf44c8c46f2 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:43.005449 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-db995152-51a0-46c5-9cad-5bf44c8c46f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84ec0b089da74156aaec034dbbac0895'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:43.006118 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-db995152-51a0-46c5-9cad-5bf44c8c46f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84ec0b089da74156aaec034dbbac0895'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:43.015136 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-db995152-51a0-46c5-9cad-5bf44c8c46f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84ec0b089da74156aaec034dbbac0895'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:43.020940 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:43.029716 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-db995152-51a0-46c5-9cad-5bf44c8c46f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84ec0b089da74156aaec034dbbac0895'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:43.037604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-db995152-51a0-46c5-9cad-5bf44c8c46f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84ec0b089da74156aaec034dbbac0895'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:43.044414 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1155/2329] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:38:42 2022] DELETE /identity/v3/users/84ec0b089da74156aaec034dbbac0895 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:43.048622 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c92b0e9-f964-45c0-96a3-7aea98b7a6e8 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:43.049105 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c92b0e9-f964-45c0-96a3-7aea98b7a6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:43.049211 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c92b0e9-f964-45c0-96a3-7aea98b7a6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:43.069702 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-9c92b0e9-f964-45c0-96a3-7aea98b7a6e8 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:38:43.070917 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9c92b0e9-f964-45c0-96a3-7aea98b7a6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:43.132338 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1175/2330] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:38:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:38:43.143410 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db995152-51a0-46c5-9cad-5bf44c8c46f2 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:43.144962 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-db995152-51a0-46c5-9cad-5bf44c8c46f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:43.203828 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db995152-51a0-46c5-9cad-5bf44c8c46f2 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:43.207439 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcd565-a2ef-4acf-8285-d5174f2536a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:43.208038 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcd565-a2ef-4acf-8285-d5174f2536a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:43.208316 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcd565-a2ef-4acf-8285-d5174f2536a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:43.208587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcd565-a2ef-4acf-8285-d5174f2536a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:43.217869 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcd565-a2ef-4acf-8285-d5174f2536a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:43.219294 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcd565-a2ef-4acf-8285-d5174f2536a6 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:43.229046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1156/2331] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:38:43.496054 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c92b0e9-f964-45c0-96a3-7aea98b7a6e8 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:43.497682 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9c92b0e9-f964-45c0-96a3-7aea98b7a6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:43.512717 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c92b0e9-f964-45c0-96a3-7aea98b7a6e8 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:43.517506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec648d-9b44-4fbd-8b3b-c39633706ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:43.518712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec648d-9b44-4fbd-8b3b-c39633706ea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:43.518997 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec648d-9b44-4fbd-8b3b-c39633706ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:43.519266 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec648d-9b44-4fbd-8b3b-c39633706ea7 admin admin] PATH_INFO: `/v3/projects/dce0c87a532c497b97e06b7f22d72ebb` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:43.520548 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ec648d-9b44-4fbd-8b3b-c39633706ea7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dce0c87a532c497b97e06b7f22d72ebb)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:43.522497 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ec648d-9b44-4fbd-8b3b-c39633706ea7 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:43.564002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:43.572567 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-86ec648d-9b44-4fbd-8b3b-c39633706ea7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dce0c87a532c497b97e06b7f22d72ebb', 'request_id': 'req-86ec648d-9b44-4fbd-8b3b-c39633706ea7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:43.578381 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-86ec648d-9b44-4fbd-8b3b-c39633706ea7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dce0c87a532c497b97e06b7f22d72ebb', 'request_id': 'req-86ec648d-9b44-4fbd-8b3b-c39633706ea7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:43.584622 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-86ec648d-9b44-4fbd-8b3b-c39633706ea7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dce0c87a532c497b97e06b7f22d72ebb', 'request_id': 'req-86ec648d-9b44-4fbd-8b3b-c39633706ea7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:43.596883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-86ec648d-9b44-4fbd-8b3b-c39633706ea7 admin admin] The token cache is being invalidate because project dce0c87a532c497b97e06b7f22d72ebb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:38:43.597169 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-86ec648d-9b44-4fbd-8b3b-c39633706ea7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:43.597781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-86ec648d-9b44-4fbd-8b3b-c39633706ea7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:38:43.599305 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1176/2332] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:38:43 2022] DELETE /identity/v3/projects/dce0c87a532c497b97e06b7f22d72ebb => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:38:46.745802 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcd565-a2ef-4acf-8285-d5174f2536a6 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:46.754260 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcd565-a2ef-4acf-8285-d5174f2536a6 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:46.800723 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a7d38e-0193-460c-a2d2-6fdeeffe572e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:46.801320 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7d38e-0193-460c-a2d2-6fdeeffe572e service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:46.801592 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7d38e-0193-460c-a2d2-6fdeeffe572e service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:46.801859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7d38e-0193-460c-a2d2-6fdeeffe572e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:46.803560 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c6a7d38e-0193-460c-a2d2-6fdeeffe572e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:46.887431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a7d38e-0193-460c-a2d2-6fdeeffe572e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:46.888916 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a7d38e-0193-460c-a2d2-6fdeeffe572e service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:46.900064 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1157/2333] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:38:53.594019 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=10,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:38:55.879165 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec648d-9b44-4fbd-8b3b-c39633706ea7 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:38:55.880860 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-86ec648d-9b44-4fbd-8b3b-c39633706ea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:55.893204 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec648d-9b44-4fbd-8b3b-c39633706ea7 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:38:55.896762 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447460bc-34e6-4610-b48e-3e035d7e03db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:38:55.897360 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447460bc-34e6-4610-b48e-3e035d7e03db service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:38:55.897639 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447460bc-34e6-4610-b48e-3e035d7e03db service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:38:55.897912 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447460bc-34e6-4610-b48e-3e035d7e03db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:38:55.899621 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-447460bc-34e6-4610-b48e-3e035d7e03db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:38:55.963488 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447460bc-34e6-4610-b48e-3e035d7e03db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:38:55.965025 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447460bc-34e6-4610-b48e-3e035d7e03db service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:38:55.975199 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1177/2334] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:38:55 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:38:56.895023 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=94,keystone:DELETE=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:39:03.358720 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a7d38e-0193-460c-a2d2-6fdeeffe572e service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:39:03.360182 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c6a7d38e-0193-460c-a2d2-6fdeeffe572e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:39:03.428526 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a7d38e-0193-460c-a2d2-6fdeeffe572e service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:39:03.433531 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f193876-7d9b-4e31-81ed-7edf2cbf59f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfbaaf32475a4c82a34948602d8f4e35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:39:03.434176 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f193876-7d9b-4e31-81ed-7edf2cbf59f6 service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:39:03.434438 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f193876-7d9b-4e31-81ed-7edf2cbf59f6 service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:39:03.434833 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f193876-7d9b-4e31-81ed-7edf2cbf59f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:39:03.436398 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1f193876-7d9b-4e31-81ed-7edf2cbf59f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:39:03.447451 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f193876-7d9b-4e31-81ed-7edf2cbf59f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:39:03.449055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f193876-7d9b-4e31-81ed-7edf2cbf59f6 service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:39:03.458755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1158/2335] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:39:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:39:05.970843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=30 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:39:13.454546 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=32 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:40:08.311852 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447460bc-34e6-4610-b48e-3e035d7e03db service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:40:08.313767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-447460bc-34e6-4610-b48e-3e035d7e03db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:40:08.332403 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447460bc-34e6-4610-b48e-3e035d7e03db service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:40:08.337819 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570ab29-82f8-4a87-8474-009be5de2bed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:40:08.338736 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570ab29-82f8-4a87-8474-009be5de2bed service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:40:08.338923 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570ab29-82f8-4a87-8474-009be5de2bed service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:40:08.339318 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570ab29-82f8-4a87-8474-009be5de2bed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:40:08.341538 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3570ab29-82f8-4a87-8474-009be5de2bed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:40:08.364140 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3570ab29-82f8-4a87-8474-009be5de2bed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:40:08.365917 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3570ab29-82f8-4a87-8474-009be5de2bed service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:40:08.379333 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1178/2336] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:40:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:40:14.628262 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f193876-7d9b-4e31-81ed-7edf2cbf59f6 service placement] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:40:14.634318 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1f193876-7d9b-4e31-81ed-7edf2cbf59f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:40:14.653642 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f193876-7d9b-4e31-81ed-7edf2cbf59f6 service placement] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:40:14.659395 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20400ef3-ed3e-4833-bde4-ab76ad05e07b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:40:14.659983 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20400ef3-ed3e-4833-bde4-ab76ad05e07b service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:40:14.660265 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20400ef3-ed3e-4833-bde4-ab76ad05e07b service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:40:14.660537 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20400ef3-ed3e-4833-bde4-ab76ad05e07b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:40:14.662374 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-20400ef3-ed3e-4833-bde4-ab76ad05e07b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:40:14.675004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20400ef3-ed3e-4833-bde4-ab76ad05e07b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:40:14.676581 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20400ef3-ed3e-4833-bde4-ab76ad05e07b service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:40:14.693280 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1159/2337] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:40:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:40:15.826154 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e34ba-e428-45c4-8a76-702b9d0e507b None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:40:15.826586 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e34ba-e428-45c4-8a76-702b9d0e507b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:40:15.826761 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e34ba-e428-45c4-8a76-702b9d0e507b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:40:15.845881 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-660e34ba-e428-45c4-8a76-702b9d0e507b None None] MFA Rules not processed for user `7a59f547a20e48ee88e3e620ba1e87f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:40:15.847076 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-660e34ba-e428-45c4-8a76-702b9d0e507b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:40:15.901223 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1179/2338] 10.222.0.20 () {58 vars in 967 bytes} [Thu Oct 20 06:40:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:40:15.941170 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20400ef3-ed3e-4833-bde4-ab76ad05e07b service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:40:15.942596 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-20400ef3-ed3e-4833-bde4-ab76ad05e07b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:40:16.009974 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20400ef3-ed3e-4833-bde4-ab76ad05e07b service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:40:16.015046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d839bd5-a3b9-44f4-a8c7-53ac84526e91 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeeab506ab5c4d6386fc899086f0abf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:40:16.015639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d839bd5-a3b9-44f4-a8c7-53ac84526e91 service swift] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:40:16.015916 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d839bd5-a3b9-44f4-a8c7-53ac84526e91 service swift] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:40:16.016186 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d839bd5-a3b9-44f4-a8c7-53ac84526e91 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:40:16.023934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d839bd5-a3b9-44f4-a8c7-53ac84526e91 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:40:16.025591 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d839bd5-a3b9-44f4-a8c7-53ac84526e91 service swift] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:40:16.034018 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1160/2339] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Oct 20 06:40:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 06:40:22.512348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e34ba-e428-45c4-8a76-702b9d0e507b None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:40:22.513933 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-660e34ba-e428-45c4-8a76-702b9d0e507b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:40:22.578252 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e34ba-e428-45c4-8a76-702b9d0e507b None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:40:22.582746 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f0634-c25e-4034-a95d-a3610113ad0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:40:22.583367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f0634-c25e-4034-a95d-a3610113ad0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:40:22.583647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f0634-c25e-4034-a95d-a3610113ad0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:40:22.583920 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f0634-c25e-4034-a95d-a3610113ad0b admin admin] PATH_INFO: `/v3/users/30be578ddd664bc79498d9a21f708f04` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:40:22.585069 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f0634-c25e-4034-a95d-a3610113ad0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=30be578ddd664bc79498d9a21f708f04)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:40:22.586989 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f0634-c25e-4034-a95d-a3610113ad0b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:40:22.631930 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b52f0634-c25e-4034-a95d-a3610113ad0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30be578ddd664bc79498d9a21f708f04'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:40:22.632933 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b52f0634-c25e-4034-a95d-a3610113ad0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30be578ddd664bc79498d9a21f708f04'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:40:22.640386 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b52f0634-c25e-4034-a95d-a3610113ad0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30be578ddd664bc79498d9a21f708f04'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:40:22.652768 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b52f0634-c25e-4034-a95d-a3610113ad0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30be578ddd664bc79498d9a21f708f04'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:40:22.654133 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b52f0634-c25e-4034-a95d-a3610113ad0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30be578ddd664bc79498d9a21f708f04'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:40:22.666376 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1180/2340] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:40:22 2022] DELETE /identity/v3/users/30be578ddd664bc79498d9a21f708f04 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:40:22.837898 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d839bd5-a3b9-44f4-a8c7-53ac84526e91 service swift] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:40:22.848919 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d839bd5-a3b9-44f4-a8c7-53ac84526e91 service swift] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:40:22.914948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0726ec-f63c-446c-856f-6b55095164f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:40:22.915596 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0726ec-f63c-446c-856f-6b55095164f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:40:22.915876 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0726ec-f63c-446c-856f-6b55095164f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:40:22.916148 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0726ec-f63c-446c-856f-6b55095164f8 admin admin] PATH_INFO: `/v3/projects/c6d602fe085a4d7d82788e61287b352d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:40:22.917293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0726ec-f63c-446c-856f-6b55095164f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6d602fe085a4d7d82788e61287b352d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:40:22.919196 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0726ec-f63c-446c-856f-6b55095164f8 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:40:22.965208 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4a0726ec-f63c-446c-856f-6b55095164f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6d602fe085a4d7d82788e61287b352d', 'request_id': 'req-4a0726ec-f63c-446c-856f-6b55095164f8'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:40:22.970939 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4a0726ec-f63c-446c-856f-6b55095164f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6d602fe085a4d7d82788e61287b352d', 'request_id': 'req-4a0726ec-f63c-446c-856f-6b55095164f8'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:40:22.977110 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4a0726ec-f63c-446c-856f-6b55095164f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6d602fe085a4d7d82788e61287b352d', 'request_id': 'req-4a0726ec-f63c-446c-856f-6b55095164f8'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:40:22.989281 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4a0726ec-f63c-446c-856f-6b55095164f8 admin admin] The token cache is being invalidate because project c6d602fe085a4d7d82788e61287b352d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:40:22.989570 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4a0726ec-f63c-446c-856f-6b55095164f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:40:22.990195 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4a0726ec-f63c-446c-856f-6b55095164f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:40:22.991693 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1161/2341] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:40:22 2022] DELETE /identity/v3/projects/c6d602fe085a4d7d82788e61287b352d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:40:22.995729 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f0634-c25e-4034-a95d-a3610113ad0b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:40:22.997915 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b52f0634-c25e-4034-a95d-a3610113ad0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:40:23.011800 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f0634-c25e-4034-a95d-a3610113ad0b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:40:23.022101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:40:23.073775 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce62ef-f92a-4313-b4a3-261a2a33a592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:40:23.074406 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce62ef-f92a-4313-b4a3-261a2a33a592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:40:23.074688 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce62ef-f92a-4313-b4a3-261a2a33a592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:40:23.075597 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce62ef-f92a-4313-b4a3-261a2a33a592 admin admin] PATH_INFO: `/v3/users/5fd3f793b3f04519bdb213f237d7d7ca` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:40:23.076754 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ce62ef-f92a-4313-b4a3-261a2a33a592 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fd3f793b3f04519bdb213f237d7d7ca)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:40:23.078665 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ce62ef-f92a-4313-b4a3-261a2a33a592 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:40:23.126344 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-41ce62ef-f92a-4313-b4a3-261a2a33a592 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fd3f793b3f04519bdb213f237d7d7ca'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:40:23.126921 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-41ce62ef-f92a-4313-b4a3-261a2a33a592 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fd3f793b3f04519bdb213f237d7d7ca'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:40:23.134590 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-41ce62ef-f92a-4313-b4a3-261a2a33a592 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fd3f793b3f04519bdb213f237d7d7ca'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:40:23.146417 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-41ce62ef-f92a-4313-b4a3-261a2a33a592 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fd3f793b3f04519bdb213f237d7d7ca'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:40:23.147043 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-41ce62ef-f92a-4313-b4a3-261a2a33a592 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fd3f793b3f04519bdb213f237d7d7ca'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:40:23.158936 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1181/2342] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:40:22 2022] DELETE /identity/v3/users/5fd3f793b3f04519bdb213f237d7d7ca => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:40:23.335203 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0726ec-f63c-446c-856f-6b55095164f8 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:40:23.336668 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-4a0726ec-f63c-446c-856f-6b55095164f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:40:23.351392 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0726ec-f63c-446c-856f-6b55095164f8 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:40:23.355380 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:40:23.413547 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a98f7-1ffc-404c-8259-88231ec8803c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:40:23.414278 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a98f7-1ffc-404c-8259-88231ec8803c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:40:23.414566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a98f7-1ffc-404c-8259-88231ec8803c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:40:23.414843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a98f7-1ffc-404c-8259-88231ec8803c admin admin] PATH_INFO: `/v3/projects/3a42dd67974d4409b4c1401e6f965e3d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:40:23.415970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a98f7-1ffc-404c-8259-88231ec8803c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a42dd67974d4409b4c1401e6f965e3d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:40:23.418437 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a98f7-1ffc-404c-8259-88231ec8803c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:40:23.467073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-320a98f7-1ffc-404c-8259-88231ec8803c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a42dd67974d4409b4c1401e6f965e3d', 'request_id': 'req-320a98f7-1ffc-404c-8259-88231ec8803c'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:40:23.472596 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-320a98f7-1ffc-404c-8259-88231ec8803c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a42dd67974d4409b4c1401e6f965e3d', 'request_id': 'req-320a98f7-1ffc-404c-8259-88231ec8803c'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:40:23.478704 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-320a98f7-1ffc-404c-8259-88231ec8803c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a42dd67974d4409b4c1401e6f965e3d', 'request_id': 'req-320a98f7-1ffc-404c-8259-88231ec8803c'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:40:23.491124 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-320a98f7-1ffc-404c-8259-88231ec8803c admin admin] The token cache is being invalidate because project 3a42dd67974d4409b4c1401e6f965e3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:40:23.491407 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-320a98f7-1ffc-404c-8259-88231ec8803c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:40:23.492017 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-320a98f7-1ffc-404c-8259-88231ec8803c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:40:23.493529 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1162/2343] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:40:23 2022] DELETE /identity/v3/projects/3a42dd67974d4409b4c1401e6f965e3d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:40:31.329492 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce62ef-f92a-4313-b4a3-261a2a33a592 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:40:31.330888 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-41ce62ef-f92a-4313-b4a3-261a2a33a592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:40:31.388682 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce62ef-f92a-4313-b4a3-261a2a33a592 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:40:31.431585 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:40:31.438028 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e808d-704e-4984-9e59-054dbabd9518 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:40:31.439229 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e808d-704e-4984-9e59-054dbabd9518 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:40:31.439644 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e808d-704e-4984-9e59-054dbabd9518 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:40:31.439946 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e808d-704e-4984-9e59-054dbabd9518 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:40:31.442192 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f23e808d-704e-4984-9e59-054dbabd9518 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:40:31.522173 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e808d-704e-4984-9e59-054dbabd9518 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:40:31.523589 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e808d-704e-4984-9e59-054dbabd9518 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:40:31.541956 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1182/2344] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:40:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:40:33.488096 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:40:41.539285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=40 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:41:12.536081 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a98f7-1ffc-404c-8259-88231ec8803c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:41:12.537495 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-320a98f7-1ffc-404c-8259-88231ec8803c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:41:12.550148 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a98f7-1ffc-404c-8259-88231ec8803c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:41:12.553646 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2534bc2-6778-4049-8fb6-352962c30a24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:41:12.554236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2534bc2-6778-4049-8fb6-352962c30a24 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:41:12.554509 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2534bc2-6778-4049-8fb6-352962c30a24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:41:12.554775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2534bc2-6778-4049-8fb6-352962c30a24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:41:12.556471 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e2534bc2-6778-4049-8fb6-352962c30a24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:41:12.627116 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2534bc2-6778-4049-8fb6-352962c30a24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:41:12.628830 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2534bc2-6778-4049-8fb6-352962c30a24 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:41:12.638692 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1163/2345] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:41:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:41:22.633727 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=33 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:13.893063 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e808d-704e-4984-9e59-054dbabd9518 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:13.894907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f23e808d-704e-4984-9e59-054dbabd9518 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:13.950438 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e808d-704e-4984-9e59-054dbabd9518 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:13.955091 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609b00a-76bd-4a92-ba0f-eca48daaaf84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:13.955698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609b00a-76bd-4a92-ba0f-eca48daaaf84 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:13.956004 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609b00a-76bd-4a92-ba0f-eca48daaaf84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:13.956277 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609b00a-76bd-4a92-ba0f-eca48daaaf84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:13.958037 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2609b00a-76bd-4a92-ba0f-eca48daaaf84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:14.022332 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609b00a-76bd-4a92-ba0f-eca48daaaf84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:14.023710 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609b00a-76bd-4a92-ba0f-eca48daaaf84 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:14.033927 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1183/2346] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:42:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:42:17.091028 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2534bc2-6778-4049-8fb6-352962c30a24 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:17.093158 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e2534bc2-6778-4049-8fb6-352962c30a24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:17.156891 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609b00a-76bd-4a92-ba0f-eca48daaaf84 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:17.158372 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2609b00a-76bd-4a92-ba0f-eca48daaaf84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:17.168147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2534bc2-6778-4049-8fb6-352962c30a24 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:17.172425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609b00a-76bd-4a92-ba0f-eca48daaaf84 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:17.173888 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad21b73-b14c-42e7-8913-15d8f6e159c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfbaaf32475a4c82a34948602d8f4e35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:17.174597 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad21b73-b14c-42e7-8913-15d8f6e159c0 service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:17.174959 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad21b73-b14c-42e7-8913-15d8f6e159c0 service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:17.175651 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad21b73-b14c-42e7-8913-15d8f6e159c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:17.177864 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-cad21b73-b14c-42e7-8913-15d8f6e159c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:17.179694 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b51e1b-3787-4d82-82cc-550dcb3e74ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:17.180336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b51e1b-3787-4d82-82cc-550dcb3e74ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:17.180612 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b51e1b-3787-4d82-82cc-550dcb3e74ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:17.180976 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b51e1b-3787-4d82-82cc-550dcb3e74ad admin admin] PATH_INFO: `/v3/users/6da8a7cb6ef849faad7d7091a1095ab9` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:17.182110 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b51e1b-3787-4d82-82cc-550dcb3e74ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=6da8a7cb6ef849faad7d7091a1095ab9)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:17.184057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b51e1b-3787-4d82-82cc-550dcb3e74ad admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:17.191036 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad21b73-b14c-42e7-8913-15d8f6e159c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:17.192644 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad21b73-b14c-42e7-8913-15d8f6e159c0 service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:17.206938 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1164/2347] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:42:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:42:17.239088 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f4b51e1b-3787-4d82-82cc-550dcb3e74ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6da8a7cb6ef849faad7d7091a1095ab9'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:17.239823 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f4b51e1b-3787-4d82-82cc-550dcb3e74ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6da8a7cb6ef849faad7d7091a1095ab9'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:17.247878 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f4b51e1b-3787-4d82-82cc-550dcb3e74ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6da8a7cb6ef849faad7d7091a1095ab9'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:17.260634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f4b51e1b-3787-4d82-82cc-550dcb3e74ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6da8a7cb6ef849faad7d7091a1095ab9'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:17.261492 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f4b51e1b-3787-4d82-82cc-550dcb3e74ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6da8a7cb6ef849faad7d7091a1095ab9'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:17.274205 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1184/2348] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:42:17 2022] DELETE /identity/v3/users/6da8a7cb6ef849faad7d7091a1095ab9 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:17.474064 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad21b73-b14c-42e7-8913-15d8f6e159c0 service placement] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:17.488727 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad21b73-b14c-42e7-8913-15d8f6e159c0 service placement] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:17.547186 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358b445-939f-474a-9f81-f16028ace567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:17.547815 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358b445-939f-474a-9f81-f16028ace567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:17.548095 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358b445-939f-474a-9f81-f16028ace567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:17.548363 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358b445-939f-474a-9f81-f16028ace567 admin admin] PATH_INFO: `/v3/projects/b21953597b4c45059f7f78ac8fdecbee` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:17.549505 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358b445-939f-474a-9f81-f16028ace567 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b21953597b4c45059f7f78ac8fdecbee)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:17.551385 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358b445-939f-474a-9f81-f16028ace567 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:17.596285 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2358b445-939f-474a-9f81-f16028ace567 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b21953597b4c45059f7f78ac8fdecbee', 'request_id': 'req-2358b445-939f-474a-9f81-f16028ace567'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:17.603011 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2358b445-939f-474a-9f81-f16028ace567 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b21953597b4c45059f7f78ac8fdecbee', 'request_id': 'req-2358b445-939f-474a-9f81-f16028ace567'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:17.609856 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2358b445-939f-474a-9f81-f16028ace567 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b21953597b4c45059f7f78ac8fdecbee', 'request_id': 'req-2358b445-939f-474a-9f81-f16028ace567'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:17.622239 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2358b445-939f-474a-9f81-f16028ace567 admin admin] The token cache is being invalidate because project b21953597b4c45059f7f78ac8fdecbee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:42:17.622530 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2358b445-939f-474a-9f81-f16028ace567 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:17.623358 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2358b445-939f-474a-9f81-f16028ace567 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:17.624907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1165/2349] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:42:17 2022] DELETE /identity/v3/projects/b21953597b4c45059f7f78ac8fdecbee => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:17.631488 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d94815-6b27-4868-9192-c4ccd5669c8d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:17.631769 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d94815-6b27-4868-9192-c4ccd5669c8d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:17.632068 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d94815-6b27-4868-9192-c4ccd5669c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:17.651304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-92d94815-6b27-4868-9192-c4ccd5669c8d None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:17.652487 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-92d94815-6b27-4868-9192-c4ccd5669c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:17.669614 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:17.713040 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1185/2350] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:17.721999 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649b287-8a05-4d16-a05d-9a9328fe1af7 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:17.722283 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649b287-8a05-4d16-a05d-9a9328fe1af7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:17.722557 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649b287-8a05-4d16-a05d-9a9328fe1af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:17.743980 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-d649b287-8a05-4d16-a05d-9a9328fe1af7 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:17.745213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d649b287-8a05-4d16-a05d-9a9328fe1af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:17.754164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1166/2351] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:17.758375 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d94815-6b27-4868-9192-c4ccd5669c8d None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:17.768870 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d94815-6b27-4868-9192-c4ccd5669c8d None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:17.773371 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea39f96-34a4-44a1-a450-98a623b03491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:17.773963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea39f96-34a4-44a1-a450-98a623b03491 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:17.774241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea39f96-34a4-44a1-a450-98a623b03491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:17.774511 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea39f96-34a4-44a1-a450-98a623b03491 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:17.775145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea39f96-34a4-44a1-a450-98a623b03491 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:42:17.775444 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea39f96-34a4-44a1-a450-98a623b03491 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:17.777073 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea39f96-34a4-44a1-a450-98a623b03491 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:17.788495 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1186/2352] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:42:17.792760 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649b287-8a05-4d16-a05d-9a9328fe1af7 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:17.802846 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649b287-8a05-4d16-a05d-9a9328fe1af7 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:17.807497 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15b3a6-09b3-4b08-8392-29f7beba4b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:17.808087 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15b3a6-09b3-4b08-8392-29f7beba4b9e admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:17.808373 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15b3a6-09b3-4b08-8392-29f7beba4b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:17.808645 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15b3a6-09b3-4b08-8392-29f7beba4b9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:17.809310 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15b3a6-09b3-4b08-8392-29f7beba4b9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:17.811927 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15b3a6-09b3-4b08-8392-29f7beba4b9e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:17.822841 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:17.840182 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1167/2353] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:42:17 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:42:17.844236 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea39f96-34a4-44a1-a450-98a623b03491 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:17.854017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea39f96-34a4-44a1-a450-98a623b03491 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:17.912959 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7acae-3e1e-4c9b-bfec-e8849510bc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:17.913563 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7acae-3e1e-4c9b-bfec-e8849510bc38 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:17.913844 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7acae-3e1e-4c9b-bfec-e8849510bc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:17.914164 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7acae-3e1e-4c9b-bfec-e8849510bc38 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:17.914782 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7acae-3e1e-4c9b-bfec-e8849510bc38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:17.916728 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7acae-3e1e-4c9b-bfec-e8849510bc38 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:17.939179 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1187/2354] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:42:17 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:42:17.942636 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15b3a6-09b3-4b08-8392-29f7beba4b9e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:17.953400 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15b3a6-09b3-4b08-8392-29f7beba4b9e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:17.958022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c1058-e4cf-4f3b-8667-36e36a74a569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:17.958617 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c1058-e4cf-4f3b-8667-36e36a74a569 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:17.958898 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c1058-e4cf-4f3b-8667-36e36a74a569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:17.959170 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c1058-e4cf-4f3b-8667-36e36a74a569 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:17.959807 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c1058-e4cf-4f3b-8667-36e36a74a569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:17.961443 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c1058-e4cf-4f3b-8667-36e36a74a569 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:17.971217 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1168/2355] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:42:17.975180 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7acae-3e1e-4c9b-bfec-e8849510bc38 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:17.987112 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7acae-3e1e-4c9b-bfec-e8849510bc38 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:17.991752 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b70214-dd9d-462d-96cc-4baa1f169ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:17.992347 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b70214-dd9d-462d-96cc-4baa1f169ac0 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:17.992624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b70214-dd9d-462d-96cc-4baa1f169ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:17.992926 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b70214-dd9d-462d-96cc-4baa1f169ac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:17.993556 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b70214-dd9d-462d-96cc-4baa1f169ac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:17.995143 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b70214-dd9d-462d-96cc-4baa1f169ac0 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:18.006487 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1188/2356] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:42:18.010945 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c1058-e4cf-4f3b-8667-36e36a74a569 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:18.022255 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c1058-e4cf-4f3b-8667-36e36a74a569 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:18.027073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d0f6ca-8dce-47d7-98dd-71413e06e910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:18.027722 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0f6ca-8dce-47d7-98dd-71413e06e910 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:18.028006 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0f6ca-8dce-47d7-98dd-71413e06e910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:18.028278 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0f6ca-8dce-47d7-98dd-71413e06e910 admin admin] PATH_INFO: `/v3/projects/d198c9c79b3148df8c65e7a7a66d18bd/users/b31ed8707ab34d9182cfaa0b8699ed22/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:18.043772 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d0f6ca-8dce-47d7-98dd-71413e06e910 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d198c9c79b3148df8c65e7a7a66d18bd, user_id=b31ed8707ab34d9182cfaa0b8699ed22, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:18.046495 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d0f6ca-8dce-47d7-98dd-71413e06e910 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:18.059166 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1169/2357] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:42:18 2022] PUT /identity/v3/projects/d198c9c79b3148df8c65e7a7a66d18bd/users/b31ed8707ab34d9182cfaa0b8699ed22/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:18.069767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4e04c-a1ec-4029-aa78-956b41d210c0 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:18.070053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4e04c-a1ec-4029-aa78-956b41d210c0 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:18.070327 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4e04c-a1ec-4029-aa78-956b41d210c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:18.097762 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-42f4e04c-a1ec-4029-aa78-956b41d210c0 None None] MFA Rules not processed for user `b31ed8707ab34d9182cfaa0b8699ed22`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:18.098962 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-42f4e04c-a1ec-4029-aa78-956b41d210c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:18.162034 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1189/2358] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:18.173590 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d0f6ca-8dce-47d7-98dd-71413e06e910 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:18.175283 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b6d0f6ca-8dce-47d7-98dd-71413e06e910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:18.246283 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d0f6ca-8dce-47d7-98dd-71413e06e910 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:18.252825 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a2682-6cc3-4a83-ac6a-07332d837d8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:18.253417 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a2682-6cc3-4a83-ac6a-07332d837d8d service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:18.253692 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a2682-6cc3-4a83-ac6a-07332d837d8d service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:18.253956 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a2682-6cc3-4a83-ac6a-07332d837d8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:18.261480 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a2682-6cc3-4a83-ac6a-07332d837d8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:18.263047 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a2682-6cc3-4a83-ac6a-07332d837d8d service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:18.272381 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1170/2359] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:42:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:42:18.429579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4e04c-a1ec-4029-aa78-956b41d210c0 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:18.431014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-42f4e04c-a1ec-4029-aa78-956b41d210c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:18.452243 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:18.490481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4e04c-a1ec-4029-aa78-956b41d210c0 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:18.494041 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3136872-6a58-4346-885a-46dd1b14c5f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:18.494643 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3136872-6a58-4346-885a-46dd1b14c5f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:18.494924 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3136872-6a58-4346-885a-46dd1b14c5f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:18.495188 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3136872-6a58-4346-885a-46dd1b14c5f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:18.496928 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b3136872-6a58-4346-885a-46dd1b14c5f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:18.510921 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3136872-6a58-4346-885a-46dd1b14c5f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:18.512330 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3136872-6a58-4346-885a-46dd1b14c5f6 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:18.523093 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1190/2360] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:42:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:42:18.601152 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a2682-6cc3-4a83-ac6a-07332d837d8d service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:18.664043 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a2682-6cc3-4a83-ac6a-07332d837d8d service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:18.668459 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db5e6c-77fd-4f86-97e7-a58ccf587cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:18.669117 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db5e6c-77fd-4f86-97e7-a58ccf587cbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:18.669400 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db5e6c-77fd-4f86-97e7-a58ccf587cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:18.670754 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db5e6c-77fd-4f86-97e7-a58ccf587cbb admin admin] PATH_INFO: `/v3/users/b31ed8707ab34d9182cfaa0b8699ed22` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:18.671879 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20db5e6c-77fd-4f86-97e7-a58ccf587cbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b31ed8707ab34d9182cfaa0b8699ed22)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:18.673834 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20db5e6c-77fd-4f86-97e7-a58ccf587cbb admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:18.717897 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:18.721412 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-20db5e6c-77fd-4f86-97e7-a58ccf587cbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b31ed8707ab34d9182cfaa0b8699ed22'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:18.722317 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-20db5e6c-77fd-4f86-97e7-a58ccf587cbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b31ed8707ab34d9182cfaa0b8699ed22'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:18.730175 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-20db5e6c-77fd-4f86-97e7-a58ccf587cbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b31ed8707ab34d9182cfaa0b8699ed22'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:18.742854 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-20db5e6c-77fd-4f86-97e7-a58ccf587cbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b31ed8707ab34d9182cfaa0b8699ed22'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:18.750909 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-20db5e6c-77fd-4f86-97e7-a58ccf587cbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b31ed8707ab34d9182cfaa0b8699ed22'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:18.756953 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1171/2361] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:42:18 2022] DELETE /identity/v3/users/b31ed8707ab34d9182cfaa0b8699ed22 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:18.761881 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2507c-6856-43f2-becd-50572bceed4a None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:18.762167 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2507c-6856-43f2-becd-50572bceed4a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:18.762439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2507c-6856-43f2-becd-50572bceed4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:18.781008 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-16b2507c-6856-43f2-becd-50572bceed4a None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:18.782202 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-16b2507c-6856-43f2-becd-50572bceed4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:18.839877 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1191/2362] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:18.850519 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db5e6c-77fd-4f86-97e7-a58ccf587cbb admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:18.851928 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-20db5e6c-77fd-4f86-97e7-a58ccf587cbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:18.909197 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db5e6c-77fd-4f86-97e7-a58ccf587cbb admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:18.912821 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29461666-1a8e-4e2a-9828-28c2d02be747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:18.913408 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29461666-1a8e-4e2a-9828-28c2d02be747 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:18.913684 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29461666-1a8e-4e2a-9828-28c2d02be747 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:18.913952 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29461666-1a8e-4e2a-9828-28c2d02be747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:18.923567 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29461666-1a8e-4e2a-9828-28c2d02be747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:18.924960 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29461666-1a8e-4e2a-9828-28c2d02be747 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:18.934112 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1172/2363] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:42:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:42:19.200503 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2507c-6856-43f2-becd-50572bceed4a None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:19.202122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-16b2507c-6856-43f2-becd-50572bceed4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:19.217764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2507c-6856-43f2-becd-50572bceed4a None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:19.223255 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7b3b0-a857-4c17-9bee-931b91d13814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:19.223934 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7b3b0-a857-4c17-9bee-931b91d13814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:19.224143 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7b3b0-a857-4c17-9bee-931b91d13814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:19.224524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7b3b0-a857-4c17-9bee-931b91d13814 admin admin] PATH_INFO: `/v3/projects/d198c9c79b3148df8c65e7a7a66d18bd` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:19.225670 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f7b3b0-a857-4c17-9bee-931b91d13814 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d198c9c79b3148df8c65e7a7a66d18bd)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:19.227559 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f7b3b0-a857-4c17-9bee-931b91d13814 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:19.275082 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c4f7b3b0-a857-4c17-9bee-931b91d13814 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd198c9c79b3148df8c65e7a7a66d18bd', 'request_id': 'req-c4f7b3b0-a857-4c17-9bee-931b91d13814'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:19.280555 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c4f7b3b0-a857-4c17-9bee-931b91d13814 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd198c9c79b3148df8c65e7a7a66d18bd', 'request_id': 'req-c4f7b3b0-a857-4c17-9bee-931b91d13814'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:19.286684 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c4f7b3b0-a857-4c17-9bee-931b91d13814 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd198c9c79b3148df8c65e7a7a66d18bd', 'request_id': 'req-c4f7b3b0-a857-4c17-9bee-931b91d13814'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:19.299791 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c4f7b3b0-a857-4c17-9bee-931b91d13814 admin admin] The token cache is being invalidate because project d198c9c79b3148df8c65e7a7a66d18bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:42:19.300161 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c4f7b3b0-a857-4c17-9bee-931b91d13814 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:19.300854 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c4f7b3b0-a857-4c17-9bee-931b91d13814 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:19.302971 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1192/2364] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:42:19 2022] DELETE /identity/v3/projects/d198c9c79b3148df8c65e7a7a66d18bd => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:19.307642 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21ae4c-ad81-4cfc-8843-6c11d458c82f None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:19.308100 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21ae4c-ad81-4cfc-8843-6c11d458c82f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:19.308190 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21ae4c-ad81-4cfc-8843-6c11d458c82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:19.328283 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-bf21ae4c-ad81-4cfc-8843-6c11d458c82f None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:19.329503 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-bf21ae4c-ad81-4cfc-8843-6c11d458c82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:19.386569 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1173/2365] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:19.395350 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26f282-2004-4925-a2d4-9dbd07edb699 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:19.395634 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26f282-2004-4925-a2d4-9dbd07edb699 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:19.395908 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26f282-2004-4925-a2d4-9dbd07edb699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:19.416856 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-6f26f282-2004-4925-a2d4-9dbd07edb699 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:19.418059 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6f26f282-2004-4925-a2d4-9dbd07edb699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:19.426449 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1193/2366] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:19.430508 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf21ae4c-ad81-4cfc-8843-6c11d458c82f None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:19.440533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf21ae4c-ad81-4cfc-8843-6c11d458c82f None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:19.445098 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d0808-3797-4f99-8c52-9c84e6656ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:19.445631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d0808-3797-4f99-8c52-9c84e6656ca9 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:19.445906 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d0808-3797-4f99-8c52-9c84e6656ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:19.446175 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d0808-3797-4f99-8c52-9c84e6656ca9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:19.446810 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d0808-3797-4f99-8c52-9c84e6656ca9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:42:19.447109 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d0808-3797-4f99-8c52-9c84e6656ca9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:19.448722 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d0808-3797-4f99-8c52-9c84e6656ca9 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:19.460477 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1174/2367] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:42:19.464593 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26f282-2004-4925-a2d4-9dbd07edb699 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:19.474516 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26f282-2004-4925-a2d4-9dbd07edb699 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:19.479773 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528a538-a3fb-4e8d-9975-83ae8473681a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:19.480377 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528a538-a3fb-4e8d-9975-83ae8473681a admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:19.480651 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528a538-a3fb-4e8d-9975-83ae8473681a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:19.480957 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528a538-a3fb-4e8d-9975-83ae8473681a admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:19.481585 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528a538-a3fb-4e8d-9975-83ae8473681a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:19.483994 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528a538-a3fb-4e8d-9975-83ae8473681a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:19.514979 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1194/2368] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:42:19 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:42:19.516094 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d0808-3797-4f99-8c52-9c84e6656ca9 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:19.526352 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d0808-3797-4f99-8c52-9c84e6656ca9 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:19.573681 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:19.585451 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d3bb9-2bcf-4e8d-bac8-ae16011dfb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:19.586246 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d3bb9-2bcf-4e8d-bac8-ae16011dfb72 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:19.586542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d3bb9-2bcf-4e8d-bac8-ae16011dfb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:19.586839 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d3bb9-2bcf-4e8d-bac8-ae16011dfb72 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:19.587457 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d3bb9-2bcf-4e8d-bac8-ae16011dfb72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:19.589449 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d3bb9-2bcf-4e8d-bac8-ae16011dfb72 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:19.611465 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1175/2369] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:42:19 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:42:19.615746 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528a538-a3fb-4e8d-9975-83ae8473681a admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:19.626164 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528a538-a3fb-4e8d-9975-83ae8473681a admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:19.631013 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f5d73-635f-40ea-8ef2-16acd2f1ffeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:19.631611 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f5d73-635f-40ea-8ef2-16acd2f1ffeb admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:19.631889 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f5d73-635f-40ea-8ef2-16acd2f1ffeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:19.632162 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f5d73-635f-40ea-8ef2-16acd2f1ffeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:19.632829 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f5d73-635f-40ea-8ef2-16acd2f1ffeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:19.634410 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f5d73-635f-40ea-8ef2-16acd2f1ffeb admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:19.644904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1195/2370] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:42:19.648767 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d3bb9-2bcf-4e8d-bac8-ae16011dfb72 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:19.659517 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d3bb9-2bcf-4e8d-bac8-ae16011dfb72 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:19.664058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c218a-8c94-489c-9cb9-cf30431a4be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:19.664633 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c218a-8c94-489c-9cb9-cf30431a4be0 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:19.664943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c218a-8c94-489c-9cb9-cf30431a4be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:19.665218 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c218a-8c94-489c-9cb9-cf30431a4be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:19.665849 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c218a-8c94-489c-9cb9-cf30431a4be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:19.667431 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c218a-8c94-489c-9cb9-cf30431a4be0 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:19.678846 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1176/2371] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:42:19.682968 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f5d73-635f-40ea-8ef2-16acd2f1ffeb admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:19.689234 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:19.693065 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f5d73-635f-40ea-8ef2-16acd2f1ffeb admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:19.699017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990a212-1c7d-40de-b6e8-ad2b03033100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:19.699798 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990a212-1c7d-40de-b6e8-ad2b03033100 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:19.700257 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990a212-1c7d-40de-b6e8-ad2b03033100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:19.700723 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990a212-1c7d-40de-b6e8-ad2b03033100 admin admin] PATH_INFO: `/v3/projects/5796255aea834fc89e6bfd5f8e2bd1e8/users/ff64f8f7a2cb4d83ae52ac95e5d84c30/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:19.715565 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2990a212-1c7d-40de-b6e8-ad2b03033100 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5796255aea834fc89e6bfd5f8e2bd1e8, user_id=ff64f8f7a2cb4d83ae52ac95e5d84c30, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:19.719085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2990a212-1c7d-40de-b6e8-ad2b03033100 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:19.731882 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1196/2372] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:42:19 2022] PUT /identity/v3/projects/5796255aea834fc89e6bfd5f8e2bd1e8/users/ff64f8f7a2cb4d83ae52ac95e5d84c30/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:19.742731 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6efed-4383-469e-bfae-c09b0238e8bd None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:19.743014 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6efed-4383-469e-bfae-c09b0238e8bd None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:19.743286 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6efed-4383-469e-bfae-c09b0238e8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:19.770207 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-46f6efed-4383-469e-bfae-c09b0238e8bd None None] MFA Rules not processed for user `ff64f8f7a2cb4d83ae52ac95e5d84c30`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:19.771388 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-46f6efed-4383-469e-bfae-c09b0238e8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:19.836474 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1177/2373] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:19.841124 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990a212-1c7d-40de-b6e8-ad2b03033100 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:19.902488 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990a212-1c7d-40de-b6e8-ad2b03033100 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:19.907758 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2cd6bb-f3cf-4bf6-8d3c-0df533249536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:19.908352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2cd6bb-f3cf-4bf6-8d3c-0df533249536 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:19.908631 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2cd6bb-f3cf-4bf6-8d3c-0df533249536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:19.908932 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2cd6bb-f3cf-4bf6-8d3c-0df533249536 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:19.909592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2cd6bb-f3cf-4bf6-8d3c-0df533249536 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:19.911487 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2cd6bb-f3cf-4bf6-8d3c-0df533249536 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:19.941325 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1197/2374] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:42:19 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:42:19.942592 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6efed-4383-469e-bfae-c09b0238e8bd None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:19.953753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6efed-4383-469e-bfae-c09b0238e8bd None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:20.008443 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05009573-33a9-4e55-9ea8-bf9fe7c0deb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:20.009067 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05009573-33a9-4e55-9ea8-bf9fe7c0deb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:20.009371 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05009573-33a9-4e55-9ea8-bf9fe7c0deb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:20.009643 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05009573-33a9-4e55-9ea8-bf9fe7c0deb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:20.010253 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05009573-33a9-4e55-9ea8-bf9fe7c0deb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:20.012141 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05009573-33a9-4e55-9ea8-bf9fe7c0deb4 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:20.036591 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1178/2375] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:42:19 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:42:20.039417 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2cd6bb-f3cf-4bf6-8d3c-0df533249536 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:20.049641 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2cd6bb-f3cf-4bf6-8d3c-0df533249536 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:20.054217 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9249a05-d2e5-44e0-b7f0-fd1a7452c73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:20.054807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9249a05-d2e5-44e0-b7f0-fd1a7452c73b admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:20.055086 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9249a05-d2e5-44e0-b7f0-fd1a7452c73b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:20.055358 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9249a05-d2e5-44e0-b7f0-fd1a7452c73b admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:20.055989 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9249a05-d2e5-44e0-b7f0-fd1a7452c73b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:20.057643 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9249a05-d2e5-44e0-b7f0-fd1a7452c73b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:20.068200 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1198/2376] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:42:20.072147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05009573-33a9-4e55-9ea8-bf9fe7c0deb4 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:20.082532 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05009573-33a9-4e55-9ea8-bf9fe7c0deb4 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:20.087298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0be4a-7c89-4238-a1e4-adac618a3562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:20.087947 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0be4a-7c89-4238-a1e4-adac618a3562 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:20.088228 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0be4a-7c89-4238-a1e4-adac618a3562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:20.088504 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0be4a-7c89-4238-a1e4-adac618a3562 admin admin] PATH_INFO: `/v3/projects/8de5fcf102554e37895d75ffa25b6b7b/users/b69b2f082041446e964a4aa160df96e0/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:20.103732 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0be4a-7c89-4238-a1e4-adac618a3562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8de5fcf102554e37895d75ffa25b6b7b, user_id=b69b2f082041446e964a4aa160df96e0, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:20.106413 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0be4a-7c89-4238-a1e4-adac618a3562 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:20.118959 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1179/2377] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:42:20 2022] PUT /identity/v3/projects/8de5fcf102554e37895d75ffa25b6b7b/users/b69b2f082041446e964a4aa160df96e0/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:20.122927 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9249a05-d2e5-44e0-b7f0-fd1a7452c73b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:20.133030 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9249a05-d2e5-44e0-b7f0-fd1a7452c73b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:20.188272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120f549-6b1a-4b31-85c0-85594b9c0578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:20.188898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120f549-6b1a-4b31-85c0-85594b9c0578 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:20.189180 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120f549-6b1a-4b31-85c0-85594b9c0578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:20.189448 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120f549-6b1a-4b31-85c0-85594b9c0578 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:20.190066 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120f549-6b1a-4b31-85c0-85594b9c0578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:20.191636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120f549-6b1a-4b31-85c0-85594b9c0578 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:20.201096 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1199/2378] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:42:20.205647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0be4a-7c89-4238-a1e4-adac618a3562 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:20.215805 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0be4a-7c89-4238-a1e4-adac618a3562 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:20.220484 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf103d-db1c-4219-a32b-d4ad8b5f1763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:20.221171 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf103d-db1c-4219-a32b-d4ad8b5f1763 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:20.221452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf103d-db1c-4219-a32b-d4ad8b5f1763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:20.221730 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf103d-db1c-4219-a32b-d4ad8b5f1763 admin admin] PATH_INFO: `/v3/projects/8de5fcf102554e37895d75ffa25b6b7b/users/b69b2f082041446e964a4aa160df96e0/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:20.223614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf103d-db1c-4219-a32b-d4ad8b5f1763 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8de5fcf102554e37895d75ffa25b6b7b, user_id=b69b2f082041446e964a4aa160df96e0, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:20.226258 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf103d-db1c-4219-a32b-d4ad8b5f1763 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:20.236889 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1180/2379] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:42:20 2022] PUT /identity/v3/projects/8de5fcf102554e37895d75ffa25b6b7b/users/b69b2f082041446e964a4aa160df96e0/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:20.246463 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ca846-b2fc-4820-ba0c-a4f3292ada2a None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:20.246751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ca846-b2fc-4820-ba0c-a4f3292ada2a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:20.247025 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ca846-b2fc-4820-ba0c-a4f3292ada2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:20.275029 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-433ca846-b2fc-4820-ba0c-a4f3292ada2a None None] MFA Rules not processed for user `b69b2f082041446e964a4aa160df96e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:20.276224 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-433ca846-b2fc-4820-ba0c-a4f3292ada2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:20.317189 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:20.346332 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1200/2380] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:20 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:20.357916 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf103d-db1c-4219-a32b-d4ad8b5f1763 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:20.359613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5adf103d-db1c-4219-a32b-d4ad8b5f1763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:20.373673 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:20.417771 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf103d-db1c-4219-a32b-d4ad8b5f1763 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:20.422053 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461522f-7913-44ff-b1bb-844441d1720f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:20.422658 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461522f-7913-44ff-b1bb-844441d1720f service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:20.422934 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461522f-7913-44ff-b1bb-844441d1720f service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:20.423203 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461522f-7913-44ff-b1bb-844441d1720f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:20.431945 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1461522f-7913-44ff-b1bb-844441d1720f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:20.433393 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1461522f-7913-44ff-b1bb-844441d1720f service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:20.442809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1181/2381] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:42:20 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:42:20.827303 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ca846-b2fc-4820-ba0c-a4f3292ada2a None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:20.888931 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ca846-b2fc-4820-ba0c-a4f3292ada2a None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:20.893651 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de800714-be5b-4071-99a8-300d9a872b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:20.894269 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de800714-be5b-4071-99a8-300d9a872b49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:20.894544 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de800714-be5b-4071-99a8-300d9a872b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:20.894814 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de800714-be5b-4071-99a8-300d9a872b49 admin admin] PATH_INFO: `/v3/users/ff64f8f7a2cb4d83ae52ac95e5d84c30` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:20.895915 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de800714-be5b-4071-99a8-300d9a872b49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff64f8f7a2cb4d83ae52ac95e5d84c30)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:20.897858 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de800714-be5b-4071-99a8-300d9a872b49 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:20.942454 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-de800714-be5b-4071-99a8-300d9a872b49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff64f8f7a2cb4d83ae52ac95e5d84c30'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:20.943094 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-de800714-be5b-4071-99a8-300d9a872b49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff64f8f7a2cb4d83ae52ac95e5d84c30'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:20.950184 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-de800714-be5b-4071-99a8-300d9a872b49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff64f8f7a2cb4d83ae52ac95e5d84c30'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:20.962547 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-de800714-be5b-4071-99a8-300d9a872b49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff64f8f7a2cb4d83ae52ac95e5d84c30'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:20.963175 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-de800714-be5b-4071-99a8-300d9a872b49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff64f8f7a2cb4d83ae52ac95e5d84c30'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:20.975222 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1201/2382] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:42:20 2022] DELETE /identity/v3/users/ff64f8f7a2cb4d83ae52ac95e5d84c30 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:20.979800 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559abb3-748e-4a9d-892f-59d984977208 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:20.980079 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559abb3-748e-4a9d-892f-59d984977208 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:20.980347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559abb3-748e-4a9d-892f-59d984977208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:20.999711 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-7559abb3-748e-4a9d-892f-59d984977208 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:21.000927 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7559abb3-748e-4a9d-892f-59d984977208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:21.062162 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1182/2383] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:21.071704 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de800714-be5b-4071-99a8-300d9a872b49 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:21.073121 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-de800714-be5b-4071-99a8-300d9a872b49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:21.129801 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de800714-be5b-4071-99a8-300d9a872b49 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:21.133413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750ed09-faaa-45ee-96bd-256ff52bb7d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:21.134042 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750ed09-faaa-45ee-96bd-256ff52bb7d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:21.134320 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750ed09-faaa-45ee-96bd-256ff52bb7d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:21.134591 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750ed09-faaa-45ee-96bd-256ff52bb7d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:21.143905 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750ed09-faaa-45ee-96bd-256ff52bb7d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:21.145358 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750ed09-faaa-45ee-96bd-256ff52bb7d3 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:21.149162 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:21.160245 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1202/2384] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:42:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:42:21.428872 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559abb3-748e-4a9d-892f-59d984977208 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:21.430351 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7559abb3-748e-4a9d-892f-59d984977208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:21.443620 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559abb3-748e-4a9d-892f-59d984977208 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:21.448638 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c90b1-9bf8-4361-8793-e2fd8b90a7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:21.449301 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c90b1-9bf8-4361-8793-e2fd8b90a7ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:21.450117 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c90b1-9bf8-4361-8793-e2fd8b90a7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:21.450401 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c90b1-9bf8-4361-8793-e2fd8b90a7ce admin admin] PATH_INFO: `/v3/projects/5796255aea834fc89e6bfd5f8e2bd1e8` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:21.452116 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c90b1-9bf8-4361-8793-e2fd8b90a7ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=5796255aea834fc89e6bfd5f8e2bd1e8)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:21.453953 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c90b1-9bf8-4361-8793-e2fd8b90a7ce admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:21.504831 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-238c90b1-9bf8-4361-8793-e2fd8b90a7ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5796255aea834fc89e6bfd5f8e2bd1e8', 'request_id': 'req-238c90b1-9bf8-4361-8793-e2fd8b90a7ce'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:21.510483 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-238c90b1-9bf8-4361-8793-e2fd8b90a7ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5796255aea834fc89e6bfd5f8e2bd1e8', 'request_id': 'req-238c90b1-9bf8-4361-8793-e2fd8b90a7ce'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:21.518170 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-238c90b1-9bf8-4361-8793-e2fd8b90a7ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5796255aea834fc89e6bfd5f8e2bd1e8', 'request_id': 'req-238c90b1-9bf8-4361-8793-e2fd8b90a7ce'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:21.530990 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-238c90b1-9bf8-4361-8793-e2fd8b90a7ce admin admin] The token cache is being invalidate because project 5796255aea834fc89e6bfd5f8e2bd1e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:42:21.531282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-238c90b1-9bf8-4361-8793-e2fd8b90a7ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:21.531902 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-238c90b1-9bf8-4361-8793-e2fd8b90a7ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:21.533499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1183/2385] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:42:21 2022] DELETE /identity/v3/projects/5796255aea834fc89e6bfd5f8e2bd1e8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:21.537441 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750ed09-faaa-45ee-96bd-256ff52bb7d3 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:21.547126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750ed09-faaa-45ee-96bd-256ff52bb7d3 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:21.603584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37403e17-8b4d-4c2a-8baa-abcf2105ffde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:21.604237 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37403e17-8b4d-4c2a-8baa-abcf2105ffde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:21.604500 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37403e17-8b4d-4c2a-8baa-abcf2105ffde admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:21.604818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37403e17-8b4d-4c2a-8baa-abcf2105ffde admin admin] PATH_INFO: `/v3/users/b69b2f082041446e964a4aa160df96e0` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:21.605950 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37403e17-8b4d-4c2a-8baa-abcf2105ffde admin admin] RBAC: Authorizing `identity:delete_user(user_id=b69b2f082041446e964a4aa160df96e0)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:21.608018 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37403e17-8b4d-4c2a-8baa-abcf2105ffde admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:21.661271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-37403e17-8b4d-4c2a-8baa-abcf2105ffde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b69b2f082041446e964a4aa160df96e0'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:21.661920 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-37403e17-8b4d-4c2a-8baa-abcf2105ffde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b69b2f082041446e964a4aa160df96e0'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:21.668776 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-37403e17-8b4d-4c2a-8baa-abcf2105ffde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b69b2f082041446e964a4aa160df96e0'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:21.680793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-37403e17-8b4d-4c2a-8baa-abcf2105ffde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b69b2f082041446e964a4aa160df96e0'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:21.681422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-37403e17-8b4d-4c2a-8baa-abcf2105ffde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b69b2f082041446e964a4aa160df96e0'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:21.693377 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1203/2386] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:42:21 2022] DELETE /identity/v3/users/b69b2f082041446e964a4aa160df96e0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:21.956595 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c90b1-9bf8-4361-8793-e2fd8b90a7ce admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:21.958039 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-238c90b1-9bf8-4361-8793-e2fd8b90a7ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:21.971031 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c90b1-9bf8-4361-8793-e2fd8b90a7ce admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:22.006135 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:22.030521 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976def7-fb24-43de-bbdc-acdd88dc5f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:22.031151 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976def7-fb24-43de-bbdc-acdd88dc5f29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:22.031426 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976def7-fb24-43de-bbdc-acdd88dc5f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:22.031696 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976def7-fb24-43de-bbdc-acdd88dc5f29 admin admin] PATH_INFO: `/v3/projects/8de5fcf102554e37895d75ffa25b6b7b` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:22.032817 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976def7-fb24-43de-bbdc-acdd88dc5f29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8de5fcf102554e37895d75ffa25b6b7b)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:22.034808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976def7-fb24-43de-bbdc-acdd88dc5f29 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:22.079518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a976def7-fb24-43de-bbdc-acdd88dc5f29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8de5fcf102554e37895d75ffa25b6b7b', 'request_id': 'req-a976def7-fb24-43de-bbdc-acdd88dc5f29'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:22.084937 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a976def7-fb24-43de-bbdc-acdd88dc5f29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8de5fcf102554e37895d75ffa25b6b7b', 'request_id': 'req-a976def7-fb24-43de-bbdc-acdd88dc5f29'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:22.090980 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a976def7-fb24-43de-bbdc-acdd88dc5f29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8de5fcf102554e37895d75ffa25b6b7b', 'request_id': 'req-a976def7-fb24-43de-bbdc-acdd88dc5f29'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:22.103694 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a976def7-fb24-43de-bbdc-acdd88dc5f29 admin admin] The token cache is being invalidate because project 8de5fcf102554e37895d75ffa25b6b7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:42:22.104169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a976def7-fb24-43de-bbdc-acdd88dc5f29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:22.104838 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a976def7-fb24-43de-bbdc-acdd88dc5f29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:22.106329 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1184/2387] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:42:21 2022] DELETE /identity/v3/projects/8de5fcf102554e37895d75ffa25b6b7b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:22.111544 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450ab64-ddbc-4145-9a08-7466ad9c269e None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:22.111872 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450ab64-ddbc-4145-9a08-7466ad9c269e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:22.112153 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450ab64-ddbc-4145-9a08-7466ad9c269e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:22.132672 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-9450ab64-ddbc-4145-9a08-7466ad9c269e None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:22.133904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9450ab64-ddbc-4145-9a08-7466ad9c269e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:22.192582 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1204/2388] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:22.201596 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff6000-5c41-4835-a25d-131de4dda9b2 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:22.201879 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff6000-5c41-4835-a25d-131de4dda9b2 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:22.202147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff6000-5c41-4835-a25d-131de4dda9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:22.221679 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-02ff6000-5c41-4835-a25d-131de4dda9b2 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:22.222856 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-02ff6000-5c41-4835-a25d-131de4dda9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:22.230507 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1185/2389] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:22.235222 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450ab64-ddbc-4145-9a08-7466ad9c269e None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:22.244928 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450ab64-ddbc-4145-9a08-7466ad9c269e None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:22.250234 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d13ba8-02f8-4abe-96bc-c135c6e9bef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:22.250821 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d13ba8-02f8-4abe-96bc-c135c6e9bef0 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:22.251101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d13ba8-02f8-4abe-96bc-c135c6e9bef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:22.251373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d13ba8-02f8-4abe-96bc-c135c6e9bef0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:22.252040 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d13ba8-02f8-4abe-96bc-c135c6e9bef0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:42:22.252603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d13ba8-02f8-4abe-96bc-c135c6e9bef0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:22.254232 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d13ba8-02f8-4abe-96bc-c135c6e9bef0 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:22.265578 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1205/2390] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:42:22.269748 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff6000-5c41-4835-a25d-131de4dda9b2 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:22.278756 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff6000-5c41-4835-a25d-131de4dda9b2 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:22.283329 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9915964-ad2f-4a35-b438-cf6fcacd8b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:22.283912 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9915964-ad2f-4a35-b438-cf6fcacd8b3c admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:22.284188 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9915964-ad2f-4a35-b438-cf6fcacd8b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:22.284455 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9915964-ad2f-4a35-b438-cf6fcacd8b3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:22.285115 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9915964-ad2f-4a35-b438-cf6fcacd8b3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:22.287003 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9915964-ad2f-4a35-b438-cf6fcacd8b3c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:22.315293 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1186/2391] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:42:22 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:42:22.319622 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d13ba8-02f8-4abe-96bc-c135c6e9bef0 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:22.325984 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:22.330038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d13ba8-02f8-4abe-96bc-c135c6e9bef0 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:22.385979 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519fef5-4fd6-43a7-b557-21c4a9d2776a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:22.386577 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519fef5-4fd6-43a7-b557-21c4a9d2776a admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:22.386856 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519fef5-4fd6-43a7-b557-21c4a9d2776a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:22.387128 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519fef5-4fd6-43a7-b557-21c4a9d2776a admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:22.387743 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3519fef5-4fd6-43a7-b557-21c4a9d2776a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:22.390209 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3519fef5-4fd6-43a7-b557-21c4a9d2776a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:22.411017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1206/2392] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:42:22 2022] POST /identity/v3/users => generated 283 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:42:22.414986 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9915964-ad2f-4a35-b438-cf6fcacd8b3c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:22.425180 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9915964-ad2f-4a35-b438-cf6fcacd8b3c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:22.429770 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d371c4-9344-4b28-a46a-ae90ad46b2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:22.430374 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d371c4-9344-4b28-a46a-ae90ad46b2e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:22.430642 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d371c4-9344-4b28-a46a-ae90ad46b2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:22.430917 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d371c4-9344-4b28-a46a-ae90ad46b2e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:22.431559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d371c4-9344-4b28-a46a-ae90ad46b2e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:22.433221 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d371c4-9344-4b28-a46a-ae90ad46b2e6 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:22.443234 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1187/2393] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:42:22.447043 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519fef5-4fd6-43a7-b557-21c4a9d2776a admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:22.457718 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519fef5-4fd6-43a7-b557-21c4a9d2776a admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:22.462222 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ac858-241e-4957-aeba-b3053609a50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:22.462805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ac858-241e-4957-aeba-b3053609a50f admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:22.463075 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ac858-241e-4957-aeba-b3053609a50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:22.463339 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ac858-241e-4957-aeba-b3053609a50f admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:22.463964 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ac858-241e-4957-aeba-b3053609a50f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:22.465545 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ac858-241e-4957-aeba-b3053609a50f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:22.476447 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1207/2394] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:42:22.480391 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d371c4-9344-4b28-a46a-ae90ad46b2e6 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:22.490261 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d371c4-9344-4b28-a46a-ae90ad46b2e6 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:22.494899 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4a7fd-dfa7-4202-95cf-6994feb4dbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:22.495566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4a7fd-dfa7-4202-95cf-6994feb4dbcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:22.495849 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4a7fd-dfa7-4202-95cf-6994feb4dbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:22.496124 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4a7fd-dfa7-4202-95cf-6994feb4dbcd admin admin] PATH_INFO: `/v3/projects/bfe9353e508d4f7187647e9fa57484b4/users/ad97681f5cfd428788c5f0770896c4b1/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:22.511702 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b4a7fd-dfa7-4202-95cf-6994feb4dbcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfe9353e508d4f7187647e9fa57484b4, user_id=ad97681f5cfd428788c5f0770896c4b1, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:22.514969 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b4a7fd-dfa7-4202-95cf-6994feb4dbcd admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:22.528211 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1188/2395] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:42:22 2022] PUT /identity/v3/projects/bfe9353e508d4f7187647e9fa57484b4/users/ad97681f5cfd428788c5f0770896c4b1/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:22.538661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b832e-307b-4f2f-ad00-1d948077a3df None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:22.539139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b832e-307b-4f2f-ad00-1d948077a3df None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:22.539340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b832e-307b-4f2f-ad00-1d948077a3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:22.567496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-6e1b832e-307b-4f2f-ad00-1d948077a3df None None] MFA Rules not processed for user `ad97681f5cfd428788c5f0770896c4b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:22.568743 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6e1b832e-307b-4f2f-ad00-1d948077a3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:22.631674 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1208/2396] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:22.636210 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4a7fd-dfa7-4202-95cf-6994feb4dbcd admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:22.698282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4a7fd-dfa7-4202-95cf-6994feb4dbcd admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:22.703585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f3720-2464-44fc-9d70-293f5ff1761d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:22.704185 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f3720-2464-44fc-9d70-293f5ff1761d admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:22.704460 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f3720-2464-44fc-9d70-293f5ff1761d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:22.704751 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f3720-2464-44fc-9d70-293f5ff1761d admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:22.705393 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f3720-2464-44fc-9d70-293f5ff1761d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:22.707257 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f3720-2464-44fc-9d70-293f5ff1761d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:22.716609 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:22.735344 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1189/2397] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:42:22 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:42:22.739312 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b832e-307b-4f2f-ad00-1d948077a3df None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:22.750575 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b832e-307b-4f2f-ad00-1d948077a3df None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:22.806010 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b3a53-c132-40e3-917b-5faafc9bd31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:22.806626 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b3a53-c132-40e3-917b-5faafc9bd31a admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:22.806904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b3a53-c132-40e3-917b-5faafc9bd31a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:22.807177 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b3a53-c132-40e3-917b-5faafc9bd31a admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:22.807808 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b3a53-c132-40e3-917b-5faafc9bd31a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:22.809759 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b3a53-c132-40e3-917b-5faafc9bd31a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:22.830670 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1209/2398] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:42:22 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:42:22.834650 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f3720-2464-44fc-9d70-293f5ff1761d admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:22.845541 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f3720-2464-44fc-9d70-293f5ff1761d admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:22.850150 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c97161-a3c5-45f0-8e39-b1893cdca174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:22.850736 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c97161-a3c5-45f0-8e39-b1893cdca174 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:22.851042 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c97161-a3c5-45f0-8e39-b1893cdca174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:22.851312 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c97161-a3c5-45f0-8e39-b1893cdca174 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:22.851938 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c97161-a3c5-45f0-8e39-b1893cdca174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:22.853552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c97161-a3c5-45f0-8e39-b1893cdca174 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:22.863555 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1190/2399] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:42:22.867511 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b3a53-c132-40e3-917b-5faafc9bd31a admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:22.873124 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:22.879599 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b3a53-c132-40e3-917b-5faafc9bd31a admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:22.884815 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b17f2-a0ff-4fd3-bfcf-9a50d52de3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:22.885430 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b17f2-a0ff-4fd3-bfcf-9a50d52de3a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:22.885699 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b17f2-a0ff-4fd3-bfcf-9a50d52de3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:22.885967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b17f2-a0ff-4fd3-bfcf-9a50d52de3a2 admin admin] PATH_INFO: `/v3/projects/de0ec69a5a264221b4aeea8952bcc9ba/users/a3eeda1591a84882b198b413c586c8ff/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:22.900312 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b17f2-a0ff-4fd3-bfcf-9a50d52de3a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de0ec69a5a264221b4aeea8952bcc9ba, user_id=a3eeda1591a84882b198b413c586c8ff, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:22.902955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b17f2-a0ff-4fd3-bfcf-9a50d52de3a2 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:22.915745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1210/2400] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:42:22 2022] PUT /identity/v3/projects/de0ec69a5a264221b4aeea8952bcc9ba/users/a3eeda1591a84882b198b413c586c8ff/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:22.920533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c97161-a3c5-45f0-8e39-b1893cdca174 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:22.930426 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c97161-a3c5-45f0-8e39-b1893cdca174 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:22.989002 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e78d5-b956-4214-88e8-f484e816f434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:22.989603 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e78d5-b956-4214-88e8-f484e816f434 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:22.989883 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e78d5-b956-4214-88e8-f484e816f434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:22.990156 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e78d5-b956-4214-88e8-f484e816f434 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:22.990789 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610e78d5-b956-4214-88e8-f484e816f434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:22.992371 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610e78d5-b956-4214-88e8-f484e816f434 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:23.002253 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1191/2401] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:42:23.006615 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b17f2-a0ff-4fd3-bfcf-9a50d52de3a2 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:23.019942 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b17f2-a0ff-4fd3-bfcf-9a50d52de3a2 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:23.025403 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56bb7f-5918-4203-86de-c2f6f73a5307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:23.026058 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56bb7f-5918-4203-86de-c2f6f73a5307 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:23.026336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56bb7f-5918-4203-86de-c2f6f73a5307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:23.026608 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56bb7f-5918-4203-86de-c2f6f73a5307 admin admin] PATH_INFO: `/v3/projects/de0ec69a5a264221b4aeea8952bcc9ba/users/a3eeda1591a84882b198b413c586c8ff/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:23.028540 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b56bb7f-5918-4203-86de-c2f6f73a5307 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de0ec69a5a264221b4aeea8952bcc9ba, user_id=a3eeda1591a84882b198b413c586c8ff, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:23.032708 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b56bb7f-5918-4203-86de-c2f6f73a5307 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:23.043048 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1211/2402] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:42:23 2022] PUT /identity/v3/projects/de0ec69a5a264221b4aeea8952bcc9ba/users/a3eeda1591a84882b198b413c586c8ff/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:23.053529 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91807e-210b-4e31-bf27-8d9bb28eca91 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:23.053814 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91807e-210b-4e31-bf27-8d9bb28eca91 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:23.054087 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91807e-210b-4e31-bf27-8d9bb28eca91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:23.082090 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-4c91807e-210b-4e31-bf27-8d9bb28eca91 None None] MFA Rules not processed for user `a3eeda1591a84882b198b413c586c8ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:23.083290 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-4c91807e-210b-4e31-bf27-8d9bb28eca91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:23.150643 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1192/2403] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:23 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:23.162420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56bb7f-5918-4203-86de-c2f6f73a5307 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:23.164127 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1b56bb7f-5918-4203-86de-c2f6f73a5307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:23.219632 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56bb7f-5918-4203-86de-c2f6f73a5307 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:23.223220 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3eaaea-d17a-4e18-86bd-4fb746f65f8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:23.223808 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3eaaea-d17a-4e18-86bd-4fb746f65f8a service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:23.224082 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3eaaea-d17a-4e18-86bd-4fb746f65f8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:23.224348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3eaaea-d17a-4e18-86bd-4fb746f65f8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:23.233636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3eaaea-d17a-4e18-86bd-4fb746f65f8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:23.235056 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3eaaea-d17a-4e18-86bd-4fb746f65f8a service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:23.244241 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1212/2404] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:42:23 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:42:23.327758 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91807e-210b-4e31-bf27-8d9bb28eca91 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:23.337348 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91807e-210b-4e31-bf27-8d9bb28eca91 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:23.340892 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebfdff8-e63a-43e4-958d-dca873e79081 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:23.342038 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebfdff8-e63a-43e4-958d-dca873e79081 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:23.342445 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebfdff8-e63a-43e4-958d-dca873e79081 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:23.342644 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebfdff8-e63a-43e4-958d-dca873e79081 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:23.397694 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebfdff8-e63a-43e4-958d-dca873e79081 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:23.399140 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebfdff8-e63a-43e4-958d-dca873e79081 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:23.402683 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:23.410314 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1193/2405] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:42:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:42:28.100141 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3eaaea-d17a-4e18-86bd-4fb746f65f8a service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:28.161349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3eaaea-d17a-4e18-86bd-4fb746f65f8a service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:28.165790 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb6a7a-1d0b-4fc0-b4d5-a6fe651f73be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:28.166401 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb6a7a-1d0b-4fc0-b4d5-a6fe651f73be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:28.166673 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb6a7a-1d0b-4fc0-b4d5-a6fe651f73be admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:28.166939 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb6a7a-1d0b-4fc0-b4d5-a6fe651f73be admin admin] PATH_INFO: `/v3/users/ad97681f5cfd428788c5f0770896c4b1` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:28.168025 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb6a7a-1d0b-4fc0-b4d5-a6fe651f73be admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad97681f5cfd428788c5f0770896c4b1)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:28.169942 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb6a7a-1d0b-4fc0-b4d5-a6fe651f73be admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:28.215584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-08bb6a7a-1d0b-4fc0-b4d5-a6fe651f73be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad97681f5cfd428788c5f0770896c4b1'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:28.216149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-08bb6a7a-1d0b-4fc0-b4d5-a6fe651f73be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad97681f5cfd428788c5f0770896c4b1'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:28.223505 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-08bb6a7a-1d0b-4fc0-b4d5-a6fe651f73be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad97681f5cfd428788c5f0770896c4b1'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:28.235198 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:28.236367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-08bb6a7a-1d0b-4fc0-b4d5-a6fe651f73be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad97681f5cfd428788c5f0770896c4b1'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:28.237092 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-08bb6a7a-1d0b-4fc0-b4d5-a6fe651f73be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad97681f5cfd428788c5f0770896c4b1'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:28.252082 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1213/2406] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:42:28 2022] DELETE /identity/v3/users/ad97681f5cfd428788c5f0770896c4b1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:28.256845 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356f105-feb8-47a6-84c2-2d2b7c83056c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:28.257296 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356f105-feb8-47a6-84c2-2d2b7c83056c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:28.257388 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356f105-feb8-47a6-84c2-2d2b7c83056c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:28.276018 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-1356f105-feb8-47a6-84c2-2d2b7c83056c None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:28.277246 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1356f105-feb8-47a6-84c2-2d2b7c83056c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:28.339718 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1194/2407] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:28.350020 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb6a7a-1d0b-4fc0-b4d5-a6fe651f73be admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:28.351455 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-08bb6a7a-1d0b-4fc0-b4d5-a6fe651f73be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:28.408353 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb6a7a-1d0b-4fc0-b4d5-a6fe651f73be admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:28.411952 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629a3b3-4a79-4ca3-9fac-180f023dee34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:28.412545 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629a3b3-4a79-4ca3-9fac-180f023dee34 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:28.412851 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629a3b3-4a79-4ca3-9fac-180f023dee34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:28.413120 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629a3b3-4a79-4ca3-9fac-180f023dee34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:28.422108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629a3b3-4a79-4ca3-9fac-180f023dee34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:28.423524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629a3b3-4a79-4ca3-9fac-180f023dee34 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:28.432789 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1214/2408] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:42:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:42:28.704053 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356f105-feb8-47a6-84c2-2d2b7c83056c None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:28.705507 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1356f105-feb8-47a6-84c2-2d2b7c83056c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:28.717289 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356f105-feb8-47a6-84c2-2d2b7c83056c None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:28.723103 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a14f0-521d-49ce-89ed-3f82631fcc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:28.723821 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a14f0-521d-49ce-89ed-3f82631fcc02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:28.724169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a14f0-521d-49ce-89ed-3f82631fcc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:28.724544 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a14f0-521d-49ce-89ed-3f82631fcc02 admin admin] PATH_INFO: `/v3/projects/bfe9353e508d4f7187647e9fa57484b4` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:28.725756 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a14f0-521d-49ce-89ed-3f82631fcc02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfe9353e508d4f7187647e9fa57484b4)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:28.727720 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a14f0-521d-49ce-89ed-3f82631fcc02 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:28.775210 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-327a14f0-521d-49ce-89ed-3f82631fcc02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfe9353e508d4f7187647e9fa57484b4', 'request_id': 'req-327a14f0-521d-49ce-89ed-3f82631fcc02'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:28.781321 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-327a14f0-521d-49ce-89ed-3f82631fcc02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfe9353e508d4f7187647e9fa57484b4', 'request_id': 'req-327a14f0-521d-49ce-89ed-3f82631fcc02'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:28.787426 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-327a14f0-521d-49ce-89ed-3f82631fcc02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfe9353e508d4f7187647e9fa57484b4', 'request_id': 'req-327a14f0-521d-49ce-89ed-3f82631fcc02'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:28.800283 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-327a14f0-521d-49ce-89ed-3f82631fcc02 admin admin] The token cache is being invalidate because project bfe9353e508d4f7187647e9fa57484b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:42:28.800565 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-327a14f0-521d-49ce-89ed-3f82631fcc02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:28.801218 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-327a14f0-521d-49ce-89ed-3f82631fcc02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:28.802703 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1195/2409] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:42:28 2022] DELETE /identity/v3/projects/bfe9353e508d4f7187647e9fa57484b4 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:28.806807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629a3b3-4a79-4ca3-9fac-180f023dee34 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:28.816245 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629a3b3-4a79-4ca3-9fac-180f023dee34 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:28.871992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93955a5-901f-4303-b2d8-b2a71a099e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:28.872615 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93955a5-901f-4303-b2d8-b2a71a099e3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:28.872925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93955a5-901f-4303-b2d8-b2a71a099e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:28.873198 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93955a5-901f-4303-b2d8-b2a71a099e3e admin admin] PATH_INFO: `/v3/users/a3eeda1591a84882b198b413c586c8ff` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:28.874659 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93955a5-901f-4303-b2d8-b2a71a099e3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3eeda1591a84882b198b413c586c8ff)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:28.876564 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93955a5-901f-4303-b2d8-b2a71a099e3e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:28.921757 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d93955a5-901f-4303-b2d8-b2a71a099e3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3eeda1591a84882b198b413c586c8ff'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:28.922406 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d93955a5-901f-4303-b2d8-b2a71a099e3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3eeda1591a84882b198b413c586c8ff'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:28.929693 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d93955a5-901f-4303-b2d8-b2a71a099e3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3eeda1591a84882b198b413c586c8ff'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:28.941809 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d93955a5-901f-4303-b2d8-b2a71a099e3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3eeda1591a84882b198b413c586c8ff'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:28.942439 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d93955a5-901f-4303-b2d8-b2a71a099e3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3eeda1591a84882b198b413c586c8ff'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:28.954971 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1215/2410] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:42:28 2022] DELETE /identity/v3/users/a3eeda1591a84882b198b413c586c8ff => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:29.221230 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a14f0-521d-49ce-89ed-3f82631fcc02 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:29.222668 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-327a14f0-521d-49ce-89ed-3f82631fcc02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:29.238078 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a14f0-521d-49ce-89ed-3f82631fcc02 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:29.294874 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf72664-857a-49f0-b69c-a1966f97b4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:29.295503 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf72664-857a-49f0-b69c-a1966f97b4f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:29.296327 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf72664-857a-49f0-b69c-a1966f97b4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:29.296760 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf72664-857a-49f0-b69c-a1966f97b4f5 admin admin] PATH_INFO: `/v3/projects/de0ec69a5a264221b4aeea8952bcc9ba` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:29.297906 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf72664-857a-49f0-b69c-a1966f97b4f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de0ec69a5a264221b4aeea8952bcc9ba)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:29.299818 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf72664-857a-49f0-b69c-a1966f97b4f5 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:29.325556 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:29.349164 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-aaf72664-857a-49f0-b69c-a1966f97b4f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de0ec69a5a264221b4aeea8952bcc9ba', 'request_id': 'req-aaf72664-857a-49f0-b69c-a1966f97b4f5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:29.354828 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-aaf72664-857a-49f0-b69c-a1966f97b4f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de0ec69a5a264221b4aeea8952bcc9ba', 'request_id': 'req-aaf72664-857a-49f0-b69c-a1966f97b4f5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:29.361761 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-aaf72664-857a-49f0-b69c-a1966f97b4f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de0ec69a5a264221b4aeea8952bcc9ba', 'request_id': 'req-aaf72664-857a-49f0-b69c-a1966f97b4f5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:29.374465 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-aaf72664-857a-49f0-b69c-a1966f97b4f5 admin admin] The token cache is being invalidate because project de0ec69a5a264221b4aeea8952bcc9ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:42:29.374757 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-aaf72664-857a-49f0-b69c-a1966f97b4f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:29.375377 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-aaf72664-857a-49f0-b69c-a1966f97b4f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:29.376908 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1196/2411] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:42:29 2022] DELETE /identity/v3/projects/de0ec69a5a264221b4aeea8952bcc9ba => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:42:29.381968 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac2287-46d3-4f30-b406-ba048323e4ad None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:29.382409 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac2287-46d3-4f30-b406-ba048323e4ad None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:29.382496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac2287-46d3-4f30-b406-ba048323e4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:29.401843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-f9ac2287-46d3-4f30-b406-ba048323e4ad None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:29.403033 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f9ac2287-46d3-4f30-b406-ba048323e4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:29.407662 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:29.464011 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1216/2412] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:29.472240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8d671-9f83-4886-80e8-5f5732f5b854 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:29.472531 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8d671-9f83-4886-80e8-5f5732f5b854 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:29.472846 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8d671-9f83-4886-80e8-5f5732f5b854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:29.493813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-68c8d671-9f83-4886-80e8-5f5732f5b854 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:29.495022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-68c8d671-9f83-4886-80e8-5f5732f5b854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:29.503901 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1197/2413] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:29.508232 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac2287-46d3-4f30-b406-ba048323e4ad None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:29.518150 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac2287-46d3-4f30-b406-ba048323e4ad None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:29.522629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d63cad-c038-441b-ba31-3adca530f94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:29.523219 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d63cad-c038-441b-ba31-3adca530f94e admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:29.523491 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d63cad-c038-441b-ba31-3adca530f94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:29.523755 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d63cad-c038-441b-ba31-3adca530f94e admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:29.524385 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d63cad-c038-441b-ba31-3adca530f94e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:42:29.524682 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d63cad-c038-441b-ba31-3adca530f94e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:29.526282 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d63cad-c038-441b-ba31-3adca530f94e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:29.538564 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1217/2414] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:42:29.542809 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8d671-9f83-4886-80e8-5f5732f5b854 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:29.552640 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8d671-9f83-4886-80e8-5f5732f5b854 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:29.557212 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0ec89-27a2-44c5-b093-d2799cff15bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:29.557811 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0ec89-27a2-44c5-b093-d2799cff15bc admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:29.558101 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0ec89-27a2-44c5-b093-d2799cff15bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:29.558368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0ec89-27a2-44c5-b093-d2799cff15bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:29.558984 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0ec89-27a2-44c5-b093-d2799cff15bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:29.560883 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0ec89-27a2-44c5-b093-d2799cff15bc admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:29.586703 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1198/2415] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:42:29 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:42:29.590803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d63cad-c038-441b-ba31-3adca530f94e admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:29.601118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d63cad-c038-441b-ba31-3adca530f94e admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:29.657855 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea695a98-6caa-4aca-9dc0-1ca19d8a6570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:29.658458 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea695a98-6caa-4aca-9dc0-1ca19d8a6570 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:29.658736 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea695a98-6caa-4aca-9dc0-1ca19d8a6570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:29.659003 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea695a98-6caa-4aca-9dc0-1ca19d8a6570 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:29.659614 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea695a98-6caa-4aca-9dc0-1ca19d8a6570 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:29.661578 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea695a98-6caa-4aca-9dc0-1ca19d8a6570 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:29.681790 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1218/2416] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:42:29 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:42:29.685907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0ec89-27a2-44c5-b093-d2799cff15bc admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:29.697773 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0ec89-27a2-44c5-b093-d2799cff15bc admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:29.703296 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d161de0-0838-4a7a-97a9-f1b3219d3677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:29.703893 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d161de0-0838-4a7a-97a9-f1b3219d3677 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:29.704167 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d161de0-0838-4a7a-97a9-f1b3219d3677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:29.704435 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d161de0-0838-4a7a-97a9-f1b3219d3677 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:29.705105 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d161de0-0838-4a7a-97a9-f1b3219d3677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:29.706680 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d161de0-0838-4a7a-97a9-f1b3219d3677 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:29.716871 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1199/2417] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:42:29.721265 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea695a98-6caa-4aca-9dc0-1ca19d8a6570 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:29.731509 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea695a98-6caa-4aca-9dc0-1ca19d8a6570 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:29.736197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0092562-2b0b-468a-941c-633aacd599d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:29.736850 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0092562-2b0b-468a-941c-633aacd599d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:29.737139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0092562-2b0b-468a-941c-633aacd599d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:29.737396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0092562-2b0b-468a-941c-633aacd599d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:29.738022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0092562-2b0b-468a-941c-633aacd599d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:29.739587 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0092562-2b0b-468a-941c-633aacd599d1 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:29.750279 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1219/2418] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:42:29.754625 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d161de0-0838-4a7a-97a9-f1b3219d3677 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:29.765094 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d161de0-0838-4a7a-97a9-f1b3219d3677 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:29.769782 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e1fc9-b827-4710-8652-c62bcd9bdf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:29.770423 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e1fc9-b827-4710-8652-c62bcd9bdf56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:29.770705 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e1fc9-b827-4710-8652-c62bcd9bdf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:29.770978 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e1fc9-b827-4710-8652-c62bcd9bdf56 admin admin] PATH_INFO: `/v3/projects/5faf3974f907409297e44ddba3d398da/users/b3c3dde831944d2a9c7e0312f4ea6e18/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:29.785775 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e1fc9-b827-4710-8652-c62bcd9bdf56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5faf3974f907409297e44ddba3d398da, user_id=b3c3dde831944d2a9c7e0312f4ea6e18, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:29.788398 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e1fc9-b827-4710-8652-c62bcd9bdf56 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:29.801169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1200/2419] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:42:29 2022] PUT /identity/v3/projects/5faf3974f907409297e44ddba3d398da/users/b3c3dde831944d2a9c7e0312f4ea6e18/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:42:29.811380 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d25f89-13c5-4981-a800-1a4c0c88cec8 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:29.811717 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d25f89-13c5-4981-a800-1a4c0c88cec8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:29.811931 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d25f89-13c5-4981-a800-1a4c0c88cec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:29.841723 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-40d25f89-13c5-4981-a800-1a4c0c88cec8 None None] MFA Rules not processed for user `b3c3dde831944d2a9c7e0312f4ea6e18`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:29.843783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-40d25f89-13c5-4981-a800-1a4c0c88cec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:29.904747 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1220/2420] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:29.916374 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e1fc9-b827-4710-8652-c62bcd9bdf56 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:29.918105 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5a2e1fc9-b827-4710-8652-c62bcd9bdf56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:29.974428 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e1fc9-b827-4710-8652-c62bcd9bdf56 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:29.978308 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808accf-a617-4c91-9ef2-ad53316b9d93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:29.978948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808accf-a617-4c91-9ef2-ad53316b9d93 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:29.979230 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808accf-a617-4c91-9ef2-ad53316b9d93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:29.979505 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808accf-a617-4c91-9ef2-ad53316b9d93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:29.991210 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b808accf-a617-4c91-9ef2-ad53316b9d93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:29.992653 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b808accf-a617-4c91-9ef2-ad53316b9d93 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:30.001915 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1201/2421] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:42:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:42:30.022804 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d25f89-13c5-4981-a800-1a4c0c88cec8 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:30.028922 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:30.091362 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d25f89-13c5-4981-a800-1a4c0c88cec8 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:30.096307 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397a7b4-2fce-4b1d-9585-43afc86b51d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:30.096963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397a7b4-2fce-4b1d-9585-43afc86b51d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:30.097242 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397a7b4-2fce-4b1d-9585-43afc86b51d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:30.097511 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397a7b4-2fce-4b1d-9585-43afc86b51d9 admin admin] PATH_INFO: `/v3/users/b3c3dde831944d2a9c7e0312f4ea6e18` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:30.098645 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397a7b4-2fce-4b1d-9585-43afc86b51d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3c3dde831944d2a9c7e0312f4ea6e18)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:30.100556 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397a7b4-2fce-4b1d-9585-43afc86b51d9 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:30.156294 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f397a7b4-2fce-4b1d-9585-43afc86b51d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3c3dde831944d2a9c7e0312f4ea6e18'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:30.157013 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f397a7b4-2fce-4b1d-9585-43afc86b51d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3c3dde831944d2a9c7e0312f4ea6e18'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:30.165154 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f397a7b4-2fce-4b1d-9585-43afc86b51d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3c3dde831944d2a9c7e0312f4ea6e18'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:30.178396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f397a7b4-2fce-4b1d-9585-43afc86b51d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3c3dde831944d2a9c7e0312f4ea6e18'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:30.179120 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f397a7b4-2fce-4b1d-9585-43afc86b51d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3c3dde831944d2a9c7e0312f4ea6e18'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:30.196128 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1221/2422] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:42:30 2022] DELETE /identity/v3/users/b3c3dde831944d2a9c7e0312f4ea6e18 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:42:30.200676 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a8515-23b7-448e-ada8-c4953e5e485a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:30.201002 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a8515-23b7-448e-ada8-c4953e5e485a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:30.201278 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a8515-23b7-448e-ada8-c4953e5e485a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:30.223652 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-235a8515-23b7-448e-ada8-c4953e5e485a None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:30.224180 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-235a8515-23b7-448e-ada8-c4953e5e485a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:30.276032 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:30.294026 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1202/2423] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:30.304829 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397a7b4-2fce-4b1d-9585-43afc86b51d9 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:30.306276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f397a7b4-2fce-4b1d-9585-43afc86b51d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:30.377545 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397a7b4-2fce-4b1d-9585-43afc86b51d9 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:30.381633 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53db708-ac88-4ccb-aa37-6246189313c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:30.382297 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53db708-ac88-4ccb-aa37-6246189313c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:30.382875 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53db708-ac88-4ccb-aa37-6246189313c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:30.382875 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53db708-ac88-4ccb-aa37-6246189313c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:30.395109 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53db708-ac88-4ccb-aa37-6246189313c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:30.396543 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53db708-ac88-4ccb-aa37-6246189313c8 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:30.406705 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1222/2424] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:42:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:42:30.714732 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a8515-23b7-448e-ada8-c4953e5e485a None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:30.716510 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-235a8515-23b7-448e-ada8-c4953e5e485a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:30.737613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a8515-23b7-448e-ada8-c4953e5e485a None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:30.744063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f9846-8778-46d1-a597-33c6aaead8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:30.745013 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f9846-8778-46d1-a597-33c6aaead8fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:30.745444 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f9846-8778-46d1-a597-33c6aaead8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:30.745718 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f9846-8778-46d1-a597-33c6aaead8fe admin admin] PATH_INFO: `/v3/projects/5faf3974f907409297e44ddba3d398da` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:30.747239 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436f9846-8778-46d1-a597-33c6aaead8fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=5faf3974f907409297e44ddba3d398da)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:30.750246 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436f9846-8778-46d1-a597-33c6aaead8fe admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:30.799259 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-436f9846-8778-46d1-a597-33c6aaead8fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5faf3974f907409297e44ddba3d398da', 'request_id': 'req-436f9846-8778-46d1-a597-33c6aaead8fe'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:30.804725 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-436f9846-8778-46d1-a597-33c6aaead8fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5faf3974f907409297e44ddba3d398da', 'request_id': 'req-436f9846-8778-46d1-a597-33c6aaead8fe'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:30.810888 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-436f9846-8778-46d1-a597-33c6aaead8fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5faf3974f907409297e44ddba3d398da', 'request_id': 'req-436f9846-8778-46d1-a597-33c6aaead8fe'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:30.823200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-436f9846-8778-46d1-a597-33c6aaead8fe admin admin] The token cache is being invalidate because project 5faf3974f907409297e44ddba3d398da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:42:30.823466 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-436f9846-8778-46d1-a597-33c6aaead8fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:30.824165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-436f9846-8778-46d1-a597-33c6aaead8fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:30.825772 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1203/2425] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:42:30 2022] DELETE /identity/v3/projects/5faf3974f907409297e44ddba3d398da => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:30.830701 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65ede4-3c4a-4df6-a576-d600c3e182b1 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:30.831008 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65ede4-3c4a-4df6-a576-d600c3e182b1 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:30.831277 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65ede4-3c4a-4df6-a576-d600c3e182b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:30.851343 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-cc65ede4-3c4a-4df6-a576-d600c3e182b1 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:30.852534 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-cc65ede4-3c4a-4df6-a576-d600c3e182b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:30.872577 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:30.912141 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1223/2426] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:30.921630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7a00c-c3f7-4604-ad9f-fcc05cd592b5 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:30.921907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7a00c-c3f7-4604-ad9f-fcc05cd592b5 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:30.922178 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7a00c-c3f7-4604-ad9f-fcc05cd592b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:30.941852 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-37b7a00c-c3f7-4604-ad9f-fcc05cd592b5 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:30.943057 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-37b7a00c-c3f7-4604-ad9f-fcc05cd592b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:30.951093 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1204/2427] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:30.955457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65ede4-3c4a-4df6-a576-d600c3e182b1 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:30.964950 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65ede4-3c4a-4df6-a576-d600c3e182b1 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:30.973201 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70411a-7be2-474a-8e9a-198930ad6309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:30.973787 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70411a-7be2-474a-8e9a-198930ad6309 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:30.974055 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70411a-7be2-474a-8e9a-198930ad6309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:30.974320 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70411a-7be2-474a-8e9a-198930ad6309 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:30.974945 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70411a-7be2-474a-8e9a-198930ad6309 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:42:30.975246 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70411a-7be2-474a-8e9a-198930ad6309 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:30.976857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70411a-7be2-474a-8e9a-198930ad6309 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:30.989601 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1224/2428] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:42:30.993624 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7a00c-c3f7-4604-ad9f-fcc05cd592b5 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:31.003381 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7a00c-c3f7-4604-ad9f-fcc05cd592b5 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:31.010167 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2e6f2-35e6-40eb-bdb7-31d5ae7ce0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:31.010759 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2e6f2-35e6-40eb-bdb7-31d5ae7ce0e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:31.011038 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2e6f2-35e6-40eb-bdb7-31d5ae7ce0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:31.011310 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2e6f2-35e6-40eb-bdb7-31d5ae7ce0e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:31.012070 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2e6f2-35e6-40eb-bdb7-31d5ae7ce0e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:31.013989 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2e6f2-35e6-40eb-bdb7-31d5ae7ce0e0 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:31.042360 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1205/2429] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:42:30 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:42:31.047068 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70411a-7be2-474a-8e9a-198930ad6309 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:31.057281 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70411a-7be2-474a-8e9a-198930ad6309 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:31.114562 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56eb2c4-83ae-490c-a61e-77341f3a5a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:31.115167 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56eb2c4-83ae-490c-a61e-77341f3a5a45 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:31.115441 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56eb2c4-83ae-490c-a61e-77341f3a5a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:31.115709 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56eb2c4-83ae-490c-a61e-77341f3a5a45 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:31.116360 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56eb2c4-83ae-490c-a61e-77341f3a5a45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:31.118325 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56eb2c4-83ae-490c-a61e-77341f3a5a45 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:31.142525 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1225/2430] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:42:31 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:42:31.143637 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2e6f2-35e6-40eb-bdb7-31d5ae7ce0e0 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:31.154003 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2e6f2-35e6-40eb-bdb7-31d5ae7ce0e0 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:31.159619 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41944687-90a7-44f2-b24b-1d0c79c1c4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:31.160240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41944687-90a7-44f2-b24b-1d0c79c1c4c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:31.160943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41944687-90a7-44f2-b24b-1d0c79c1c4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:31.161226 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41944687-90a7-44f2-b24b-1d0c79c1c4c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:31.161871 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41944687-90a7-44f2-b24b-1d0c79c1c4c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:31.163584 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41944687-90a7-44f2-b24b-1d0c79c1c4c3 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:31.173772 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1206/2431] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:42:31.177844 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56eb2c4-83ae-490c-a61e-77341f3a5a45 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:31.187826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56eb2c4-83ae-490c-a61e-77341f3a5a45 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:31.192370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11715eb4-9bae-4106-bdc2-84eb3cc3f7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:31.193004 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11715eb4-9bae-4106-bdc2-84eb3cc3f7ac admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:31.193276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11715eb4-9bae-4106-bdc2-84eb3cc3f7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:31.193543 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11715eb4-9bae-4106-bdc2-84eb3cc3f7ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:31.194170 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11715eb4-9bae-4106-bdc2-84eb3cc3f7ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:31.195760 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11715eb4-9bae-4106-bdc2-84eb3cc3f7ac admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:31.206650 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1226/2432] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:42:31.210744 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41944687-90a7-44f2-b24b-1d0c79c1c4c3 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:31.221343 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41944687-90a7-44f2-b24b-1d0c79c1c4c3 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:31.225940 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cfeb71-e934-4999-b0d7-c805a05f7cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:31.227220 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfeb71-e934-4999-b0d7-c805a05f7cc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:31.227504 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfeb71-e934-4999-b0d7-c805a05f7cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:31.227778 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfeb71-e934-4999-b0d7-c805a05f7cc9 admin admin] PATH_INFO: `/v3/projects/7487a2c25bca4126a705b0e8d774d96f/users/dca7f029bdc940d1938203d138029581/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:31.242144 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cfeb71-e934-4999-b0d7-c805a05f7cc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7487a2c25bca4126a705b0e8d774d96f, user_id=dca7f029bdc940d1938203d138029581, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:31.244813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cfeb71-e934-4999-b0d7-c805a05f7cc9 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:31.257633 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1207/2433] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:42:31 2022] PUT /identity/v3/projects/7487a2c25bca4126a705b0e8d774d96f/users/dca7f029bdc940d1938203d138029581/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:31.268292 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260aab2-76f4-46a9-a270-62af0d81a925 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:31.268574 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260aab2-76f4-46a9-a270-62af0d81a925 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:31.268902 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260aab2-76f4-46a9-a270-62af0d81a925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:31.296532 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a260aab2-76f4-46a9-a270-62af0d81a925 None None] MFA Rules not processed for user `dca7f029bdc940d1938203d138029581`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:31.297771 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a260aab2-76f4-46a9-a270-62af0d81a925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:31.363172 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1227/2434] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:31.367637 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cfeb71-e934-4999-b0d7-c805a05f7cc9 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:31.428199 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cfeb71-e934-4999-b0d7-c805a05f7cc9 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:31.432663 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2819d-fd16-4f84-a85c-5838c892be52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:31.433298 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2819d-fd16-4f84-a85c-5838c892be52 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:31.433577 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2819d-fd16-4f84-a85c-5838c892be52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:31.433847 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2819d-fd16-4f84-a85c-5838c892be52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:31.434465 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf2819d-fd16-4f84-a85c-5838c892be52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:31.436402 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf2819d-fd16-4f84-a85c-5838c892be52 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:31.451198 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:31.464285 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1208/2435] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:42:31 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:42:31.468618 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260aab2-76f4-46a9-a270-62af0d81a925 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:31.479344 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260aab2-76f4-46a9-a270-62af0d81a925 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:31.497599 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:31.536650 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9bd863-4ece-4c5d-99c9-416696b30cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:31.537283 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9bd863-4ece-4c5d-99c9-416696b30cda admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:31.537562 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9bd863-4ece-4c5d-99c9-416696b30cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:31.537835 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9bd863-4ece-4c5d-99c9-416696b30cda admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:31.538463 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9bd863-4ece-4c5d-99c9-416696b30cda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:31.540356 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9bd863-4ece-4c5d-99c9-416696b30cda admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:31.560944 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1228/2436] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:42:31 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:42:31.564991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2819d-fd16-4f84-a85c-5838c892be52 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:31.575030 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2819d-fd16-4f84-a85c-5838c892be52 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:31.579535 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760837da-72e4-4ce4-835c-545d5be50b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:31.580123 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760837da-72e4-4ce4-835c-545d5be50b31 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:31.580399 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760837da-72e4-4ce4-835c-545d5be50b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:31.580668 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760837da-72e4-4ce4-835c-545d5be50b31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:31.581330 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760837da-72e4-4ce4-835c-545d5be50b31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:31.582928 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760837da-72e4-4ce4-835c-545d5be50b31 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:31.593716 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1209/2437] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:42:31.597561 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9bd863-4ece-4c5d-99c9-416696b30cda admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:31.607384 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9bd863-4ece-4c5d-99c9-416696b30cda admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:31.612487 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9b45d-e137-478b-aad0-ae4364e4d9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:31.613122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9b45d-e137-478b-aad0-ae4364e4d9ca admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:31.613404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9b45d-e137-478b-aad0-ae4364e4d9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:31.613678 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9b45d-e137-478b-aad0-ae4364e4d9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:31.614315 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb9b45d-e137-478b-aad0-ae4364e4d9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:31.615895 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb9b45d-e137-478b-aad0-ae4364e4d9ca admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:31.626282 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1229/2438] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:42:31.630338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760837da-72e4-4ce4-835c-545d5be50b31 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:31.640594 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760837da-72e4-4ce4-835c-545d5be50b31 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:31.645477 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c30f9-da7d-42eb-92d5-c1177fbb1723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:31.646122 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c30f9-da7d-42eb-92d5-c1177fbb1723 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:31.646404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c30f9-da7d-42eb-92d5-c1177fbb1723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:31.646678 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c30f9-da7d-42eb-92d5-c1177fbb1723 admin admin] PATH_INFO: `/v3/projects/4dcfebd184a64670a9168115e45b407d/users/e6fba3fb75514d43bb6ff52ea0504410/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:31.662467 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542c30f9-da7d-42eb-92d5-c1177fbb1723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dcfebd184a64670a9168115e45b407d, user_id=e6fba3fb75514d43bb6ff52ea0504410, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:31.665151 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542c30f9-da7d-42eb-92d5-c1177fbb1723 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:31.677883 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1210/2439] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:42:31 2022] PUT /identity/v3/projects/4dcfebd184a64670a9168115e45b407d/users/e6fba3fb75514d43bb6ff52ea0504410/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:42:31.688825 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7170ca-44b0-4e21-bf6f-ce7ce4338f1d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:31.689055 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7170ca-44b0-4e21-bf6f-ce7ce4338f1d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:31.689329 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7170ca-44b0-4e21-bf6f-ce7ce4338f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:31.718881 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-aa7170ca-44b0-4e21-bf6f-ce7ce4338f1d None None] MFA Rules not processed for user `e6fba3fb75514d43bb6ff52ea0504410`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:31.720079 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-aa7170ca-44b0-4e21-bf6f-ce7ce4338f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:31.784428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1230/2440] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:31.788888 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c30f9-da7d-42eb-92d5-c1177fbb1723 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:31.850063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c30f9-da7d-42eb-92d5-c1177fbb1723 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:31.854708 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb875e-a731-4d9c-af48-da932c08d1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:31.855302 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb875e-a731-4d9c-af48-da932c08d1b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:31.855584 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb875e-a731-4d9c-af48-da932c08d1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:31.855857 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb875e-a731-4d9c-af48-da932c08d1b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:31.856471 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb875e-a731-4d9c-af48-da932c08d1b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:31.858732 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb875e-a731-4d9c-af48-da932c08d1b8 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:31.885033 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1211/2441] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:42:31 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:42:31.889920 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7170ca-44b0-4e21-bf6f-ce7ce4338f1d None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:31.900345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7170ca-44b0-4e21-bf6f-ce7ce4338f1d None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:31.956169 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc77d9e9-a9cb-4799-a4a8-ce78773a8bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:31.956719 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77d9e9-a9cb-4799-a4a8-ce78773a8bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:31.957009 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77d9e9-a9cb-4799-a4a8-ce78773a8bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:31.957284 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77d9e9-a9cb-4799-a4a8-ce78773a8bc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:31.957900 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc77d9e9-a9cb-4799-a4a8-ce78773a8bc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:31.959821 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc77d9e9-a9cb-4799-a4a8-ce78773a8bc1 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:31.984183 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1231/2442] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:42:31 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:42:31.985557 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb875e-a731-4d9c-af48-da932c08d1b8 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:31.995880 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb875e-a731-4d9c-af48-da932c08d1b8 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:32.001134 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce7804-9012-434a-a13f-8aded6e317f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:32.001722 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce7804-9012-434a-a13f-8aded6e317f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:32.001995 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce7804-9012-434a-a13f-8aded6e317f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:32.002530 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce7804-9012-434a-a13f-8aded6e317f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:32.003158 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce7804-9012-434a-a13f-8aded6e317f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:32.004763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce7804-9012-434a-a13f-8aded6e317f2 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:32.015057 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1212/2443] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:42:32.019191 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc77d9e9-a9cb-4799-a4a8-ce78773a8bc1 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:32.029513 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc77d9e9-a9cb-4799-a4a8-ce78773a8bc1 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:32.034136 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b2230-b4c0-471b-842d-1d0deeed2430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:32.034778 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b2230-b4c0-471b-842d-1d0deeed2430 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:32.035053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b2230-b4c0-471b-842d-1d0deeed2430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:32.035348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b2230-b4c0-471b-842d-1d0deeed2430 admin admin] PATH_INFO: `/v3/projects/a3ff259aeb4547f3b94f6af7aa4a38cf/users/df6918fc73894956a7275d991ed7712b/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:32.050342 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b2230-b4c0-471b-842d-1d0deeed2430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3ff259aeb4547f3b94f6af7aa4a38cf, user_id=df6918fc73894956a7275d991ed7712b, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:32.053013 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b2230-b4c0-471b-842d-1d0deeed2430 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:32.055931 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:32.067116 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1232/2444] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:42:32 2022] PUT /identity/v3/projects/a3ff259aeb4547f3b94f6af7aa4a38cf/users/df6918fc73894956a7275d991ed7712b/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:32.071016 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce7804-9012-434a-a13f-8aded6e317f2 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:32.081009 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce7804-9012-434a-a13f-8aded6e317f2 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:32.135704 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810fe06-5dee-4214-8ec1-d8bddbf7e295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:32.136310 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810fe06-5dee-4214-8ec1-d8bddbf7e295 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:32.136589 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810fe06-5dee-4214-8ec1-d8bddbf7e295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:32.136939 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810fe06-5dee-4214-8ec1-d8bddbf7e295 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:32.137716 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7810fe06-5dee-4214-8ec1-d8bddbf7e295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:32.140324 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7810fe06-5dee-4214-8ec1-d8bddbf7e295 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:32.150071 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1213/2445] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:42:32.153964 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b2230-b4c0-471b-842d-1d0deeed2430 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:32.164970 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b2230-b4c0-471b-842d-1d0deeed2430 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:32.169554 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d32f3-c79b-40c7-a882-22f28d11b59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:32.170204 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d32f3-c79b-40c7-a882-22f28d11b59c admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:32.170525 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d32f3-c79b-40c7-a882-22f28d11b59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:32.170796 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d32f3-c79b-40c7-a882-22f28d11b59c admin admin] PATH_INFO: `/v3/projects/a3ff259aeb4547f3b94f6af7aa4a38cf/users/df6918fc73894956a7275d991ed7712b/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:32.172794 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d32f3-c79b-40c7-a882-22f28d11b59c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3ff259aeb4547f3b94f6af7aa4a38cf, user_id=df6918fc73894956a7275d991ed7712b, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:32.175411 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d32f3-c79b-40c7-a882-22f28d11b59c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:32.185372 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1233/2446] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:42:32 2022] PUT /identity/v3/projects/a3ff259aeb4547f3b94f6af7aa4a38cf/users/df6918fc73894956a7275d991ed7712b/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:32.196481 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7adcc6c-fbd3-4202-89c5-03715ca93a6b None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:32.196802 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7adcc6c-fbd3-4202-89c5-03715ca93a6b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:32.197080 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7adcc6c-fbd3-4202-89c5-03715ca93a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:32.225994 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c7adcc6c-fbd3-4202-89c5-03715ca93a6b None None] MFA Rules not processed for user `df6918fc73894956a7275d991ed7712b`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:32.227190 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c7adcc6c-fbd3-4202-89c5-03715ca93a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:32.246049 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:32.296722 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1214/2447] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:32 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:32.307921 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d32f3-c79b-40c7-a882-22f28d11b59c admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:32.309628 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7c7d32f3-c79b-40c7-a882-22f28d11b59c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:32.366927 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d32f3-c79b-40c7-a882-22f28d11b59c admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:32.370501 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc74695-29db-4160-95a4-358f33a128c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:32.371088 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc74695-29db-4160-95a4-358f33a128c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:32.371363 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc74695-29db-4160-95a4-358f33a128c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:32.371629 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc74695-29db-4160-95a4-358f33a128c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:32.428182 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc74695-29db-4160-95a4-358f33a128c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:32.429647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc74695-29db-4160-95a4-358f33a128c0 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:32.438317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1234/2448] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:42:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:42:33.940892 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7adcc6c-fbd3-4202-89c5-03715ca93a6b None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:33.950578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7adcc6c-fbd3-4202-89c5-03715ca93a6b None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:33.954304 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c97d3-5397-407e-b0ba-eeefdcb38083 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:33.954900 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c97d3-5397-407e-b0ba-eeefdcb38083 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:33.955174 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c97d3-5397-407e-b0ba-eeefdcb38083 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:33.955447 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c97d3-5397-407e-b0ba-eeefdcb38083 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:34.011488 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c97d3-5397-407e-b0ba-eeefdcb38083 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:34.012955 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c97d3-5397-407e-b0ba-eeefdcb38083 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:34.022398 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1215/2449] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:42:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:42:34.553925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc74695-29db-4160-95a4-358f33a128c0 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:34.563015 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc74695-29db-4160-95a4-358f33a128c0 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:34.566635 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e48453-3755-42e0-b6cf-8be67966dde4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:34.567232 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e48453-3755-42e0-b6cf-8be67966dde4 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:34.567508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e48453-3755-42e0-b6cf-8be67966dde4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:34.567778 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e48453-3755-42e0-b6cf-8be67966dde4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:34.576472 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e48453-3755-42e0-b6cf-8be67966dde4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:34.578389 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e48453-3755-42e0-b6cf-8be67966dde4 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:34.587759 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1235/2450] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:42:34 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:42:38.449880 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c97d3-5397-407e-b0ba-eeefdcb38083 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:38.511955 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c97d3-5397-407e-b0ba-eeefdcb38083 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:38.517125 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61c756-3665-4a2c-83e4-1d544ebc479f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:38.517744 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61c756-3665-4a2c-83e4-1d544ebc479f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:38.518051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61c756-3665-4a2c-83e4-1d544ebc479f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:38.518328 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61c756-3665-4a2c-83e4-1d544ebc479f admin admin] PATH_INFO: `/v3/users/dca7f029bdc940d1938203d138029581` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:38.519451 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c61c756-3665-4a2c-83e4-1d544ebc479f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dca7f029bdc940d1938203d138029581)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:38.522867 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c61c756-3665-4a2c-83e4-1d544ebc479f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:38.568481 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5c61c756-3665-4a2c-83e4-1d544ebc479f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dca7f029bdc940d1938203d138029581'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:38.569170 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5c61c756-3665-4a2c-83e4-1d544ebc479f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dca7f029bdc940d1938203d138029581'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:38.578052 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5c61c756-3665-4a2c-83e4-1d544ebc479f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dca7f029bdc940d1938203d138029581'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:38.580287 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:38.597145 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5c61c756-3665-4a2c-83e4-1d544ebc479f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dca7f029bdc940d1938203d138029581'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:38.605808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5c61c756-3665-4a2c-83e4-1d544ebc479f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dca7f029bdc940d1938203d138029581'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:38.615073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1216/2451] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:42:38 2022] DELETE /identity/v3/users/dca7f029bdc940d1938203d138029581 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:38.619494 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a010ac-27b6-4da8-99e1-43b071957339 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:38.619989 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a010ac-27b6-4da8-99e1-43b071957339 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:38.620091 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a010ac-27b6-4da8-99e1-43b071957339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:38.639579 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-86a010ac-27b6-4da8-99e1-43b071957339 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:38.640821 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-86a010ac-27b6-4da8-99e1-43b071957339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:38.699346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1236/2452] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:38.710267 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61c756-3665-4a2c-83e4-1d544ebc479f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:38.711686 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5c61c756-3665-4a2c-83e4-1d544ebc479f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:38.768992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61c756-3665-4a2c-83e4-1d544ebc479f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:38.772608 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568485cd-c228-4ba5-81ca-14b9f1a98da8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:38.773240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568485cd-c228-4ba5-81ca-14b9f1a98da8 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:38.773519 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568485cd-c228-4ba5-81ca-14b9f1a98da8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:38.773792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568485cd-c228-4ba5-81ca-14b9f1a98da8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:38.782904 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568485cd-c228-4ba5-81ca-14b9f1a98da8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:38.784332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568485cd-c228-4ba5-81ca-14b9f1a98da8 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:38.793687 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1217/2453] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:42:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:42:39.060515 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a010ac-27b6-4da8-99e1-43b071957339 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:39.062001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-86a010ac-27b6-4da8-99e1-43b071957339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:39.074642 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a010ac-27b6-4da8-99e1-43b071957339 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:39.083463 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4968132-6ba8-4ce5-a409-d64a2e678317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:39.084134 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4968132-6ba8-4ce5-a409-d64a2e678317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:39.084407 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4968132-6ba8-4ce5-a409-d64a2e678317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:39.084684 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4968132-6ba8-4ce5-a409-d64a2e678317 admin admin] PATH_INFO: `/v3/projects/7487a2c25bca4126a705b0e8d774d96f` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:39.085823 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4968132-6ba8-4ce5-a409-d64a2e678317 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7487a2c25bca4126a705b0e8d774d96f)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:39.088496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4968132-6ba8-4ce5-a409-d64a2e678317 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:39.110008 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:39.136829 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f4968132-6ba8-4ce5-a409-d64a2e678317 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7487a2c25bca4126a705b0e8d774d96f', 'request_id': 'req-f4968132-6ba8-4ce5-a409-d64a2e678317'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:39.143125 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f4968132-6ba8-4ce5-a409-d64a2e678317 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7487a2c25bca4126a705b0e8d774d96f', 'request_id': 'req-f4968132-6ba8-4ce5-a409-d64a2e678317'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:39.149530 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f4968132-6ba8-4ce5-a409-d64a2e678317 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7487a2c25bca4126a705b0e8d774d96f', 'request_id': 'req-f4968132-6ba8-4ce5-a409-d64a2e678317'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:39.162413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f4968132-6ba8-4ce5-a409-d64a2e678317 admin admin] The token cache is being invalidate because project 7487a2c25bca4126a705b0e8d774d96f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:42:39.162702 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f4968132-6ba8-4ce5-a409-d64a2e678317 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:39.163320 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f4968132-6ba8-4ce5-a409-d64a2e678317 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:39.164873 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1237/2454] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:42:39 2022] DELETE /identity/v3/projects/7487a2c25bca4126a705b0e8d774d96f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:39.168837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568485cd-c228-4ba5-81ca-14b9f1a98da8 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:39.178601 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568485cd-c228-4ba5-81ca-14b9f1a98da8 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:39.234430 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc882816-6b4a-4fdf-92ea-39c1da6b8f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:39.235217 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc882816-6b4a-4fdf-92ea-39c1da6b8f2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:39.235558 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc882816-6b4a-4fdf-92ea-39c1da6b8f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:39.235915 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc882816-6b4a-4fdf-92ea-39c1da6b8f2f admin admin] PATH_INFO: `/v3/users/e6fba3fb75514d43bb6ff52ea0504410` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:39.237066 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc882816-6b4a-4fdf-92ea-39c1da6b8f2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6fba3fb75514d43bb6ff52ea0504410)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:39.238971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc882816-6b4a-4fdf-92ea-39c1da6b8f2f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:39.283671 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-bc882816-6b4a-4fdf-92ea-39c1da6b8f2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6fba3fb75514d43bb6ff52ea0504410'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:39.284299 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-bc882816-6b4a-4fdf-92ea-39c1da6b8f2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6fba3fb75514d43bb6ff52ea0504410'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:39.291123 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-bc882816-6b4a-4fdf-92ea-39c1da6b8f2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6fba3fb75514d43bb6ff52ea0504410'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:39.307907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-bc882816-6b4a-4fdf-92ea-39c1da6b8f2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6fba3fb75514d43bb6ff52ea0504410'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:39.313713 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:39.315742 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-bc882816-6b4a-4fdf-92ea-39c1da6b8f2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6fba3fb75514d43bb6ff52ea0504410'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:39.323147 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1218/2455] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:42:39 2022] DELETE /identity/v3/users/e6fba3fb75514d43bb6ff52ea0504410 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:39.587263 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4968132-6ba8-4ce5-a409-d64a2e678317 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:39.588746 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f4968132-6ba8-4ce5-a409-d64a2e678317 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:39.609928 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4968132-6ba8-4ce5-a409-d64a2e678317 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:39.676247 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945f851-9f97-4dc0-a2ec-9ea1a2a8e0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:39.676912 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945f851-9f97-4dc0-a2ec-9ea1a2a8e0d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:39.677187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945f851-9f97-4dc0-a2ec-9ea1a2a8e0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:39.677488 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945f851-9f97-4dc0-a2ec-9ea1a2a8e0d7 admin admin] PATH_INFO: `/v3/projects/4dcfebd184a64670a9168115e45b407d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:39.678596 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945f851-9f97-4dc0-a2ec-9ea1a2a8e0d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dcfebd184a64670a9168115e45b407d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:39.680504 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945f851-9f97-4dc0-a2ec-9ea1a2a8e0d7 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:39.727097 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8945f851-9f97-4dc0-a2ec-9ea1a2a8e0d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dcfebd184a64670a9168115e45b407d', 'request_id': 'req-8945f851-9f97-4dc0-a2ec-9ea1a2a8e0d7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:39.733141 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8945f851-9f97-4dc0-a2ec-9ea1a2a8e0d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dcfebd184a64670a9168115e45b407d', 'request_id': 'req-8945f851-9f97-4dc0-a2ec-9ea1a2a8e0d7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:39.740070 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8945f851-9f97-4dc0-a2ec-9ea1a2a8e0d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dcfebd184a64670a9168115e45b407d', 'request_id': 'req-8945f851-9f97-4dc0-a2ec-9ea1a2a8e0d7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:39.751909 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8945f851-9f97-4dc0-a2ec-9ea1a2a8e0d7 admin admin] The token cache is being invalidate because project 4dcfebd184a64670a9168115e45b407d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:42:39.752445 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8945f851-9f97-4dc0-a2ec-9ea1a2a8e0d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:39.753078 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8945f851-9f97-4dc0-a2ec-9ea1a2a8e0d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:39.754544 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1238/2456] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:42:39 2022] DELETE /identity/v3/projects/4dcfebd184a64670a9168115e45b407d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:39.758491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc882816-6b4a-4fdf-92ea-39c1da6b8f2f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:39.759930 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-bc882816-6b4a-4fdf-92ea-39c1da6b8f2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:39.772370 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc882816-6b4a-4fdf-92ea-39c1da6b8f2f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:39.829706 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097906f-e381-43f7-ac5d-2a3d35fef698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:39.830404 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097906f-e381-43f7-ac5d-2a3d35fef698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:39.830633 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097906f-e381-43f7-ac5d-2a3d35fef698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:39.830907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097906f-e381-43f7-ac5d-2a3d35fef698 admin admin] PATH_INFO: `/v3/users/df6918fc73894956a7275d991ed7712b` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:39.832024 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097906f-e381-43f7-ac5d-2a3d35fef698 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df6918fc73894956a7275d991ed7712b)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:39.833991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097906f-e381-43f7-ac5d-2a3d35fef698 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:39.879867 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8097906f-e381-43f7-ac5d-2a3d35fef698 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df6918fc73894956a7275d991ed7712b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:39.880499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8097906f-e381-43f7-ac5d-2a3d35fef698 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df6918fc73894956a7275d991ed7712b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:39.887548 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8097906f-e381-43f7-ac5d-2a3d35fef698 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df6918fc73894956a7275d991ed7712b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:39.900326 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8097906f-e381-43f7-ac5d-2a3d35fef698 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df6918fc73894956a7275d991ed7712b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:39.907200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8097906f-e381-43f7-ac5d-2a3d35fef698 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df6918fc73894956a7275d991ed7712b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:39.912970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1219/2457] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:42:39 2022] DELETE /identity/v3/users/df6918fc73894956a7275d991ed7712b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:40.178417 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945f851-9f97-4dc0-a2ec-9ea1a2a8e0d7 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:40.179843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8945f851-9f97-4dc0-a2ec-9ea1a2a8e0d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:40.193993 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945f851-9f97-4dc0-a2ec-9ea1a2a8e0d7 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:40.228069 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:40.253587 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df4a21-2643-4f33-829d-df242ef530a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:40.254191 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df4a21-2643-4f33-829d-df242ef530a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:40.254465 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df4a21-2643-4f33-829d-df242ef530a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:40.254735 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df4a21-2643-4f33-829d-df242ef530a8 admin admin] PATH_INFO: `/v3/projects/a3ff259aeb4547f3b94f6af7aa4a38cf` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:40.255827 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df4a21-2643-4f33-829d-df242ef530a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3ff259aeb4547f3b94f6af7aa4a38cf)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:40.257767 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df4a21-2643-4f33-829d-df242ef530a8 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:40.301745 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-30df4a21-2643-4f33-829d-df242ef530a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3ff259aeb4547f3b94f6af7aa4a38cf', 'request_id': 'req-30df4a21-2643-4f33-829d-df242ef530a8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:40.310155 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-30df4a21-2643-4f33-829d-df242ef530a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3ff259aeb4547f3b94f6af7aa4a38cf', 'request_id': 'req-30df4a21-2643-4f33-829d-df242ef530a8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:40.317540 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-30df4a21-2643-4f33-829d-df242ef530a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3ff259aeb4547f3b94f6af7aa4a38cf', 'request_id': 'req-30df4a21-2643-4f33-829d-df242ef530a8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:40.330817 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-30df4a21-2643-4f33-829d-df242ef530a8 admin admin] The token cache is being invalidate because project a3ff259aeb4547f3b94f6af7aa4a38cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:42:40.331110 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-30df4a21-2643-4f33-829d-df242ef530a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:40.331723 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-30df4a21-2643-4f33-829d-df242ef530a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:42:40.333474 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1239/2458] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:42:40 2022] DELETE /identity/v3/projects/a3ff259aeb4547f3b94f6af7aa4a38cf => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:40.338326 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feba5c-36f3-4e96-b205-0ea87654c585 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:40.338618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feba5c-36f3-4e96-b205-0ea87654c585 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:40.338879 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feba5c-36f3-4e96-b205-0ea87654c585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:40.357991 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f9feba5c-36f3-4e96-b205-0ea87654c585 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:40.359090 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f9feba5c-36f3-4e96-b205-0ea87654c585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:40.416502 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1220/2459] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:40.424941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cce053-3608-4c86-8476-3cda0f3f7e29 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:40.425225 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cce053-3608-4c86-8476-3cda0f3f7e29 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:40.425524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cce053-3608-4c86-8476-3cda0f3f7e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:40.445528 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-88cce053-3608-4c86-8476-3cda0f3f7e29 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:40.446707 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-88cce053-3608-4c86-8476-3cda0f3f7e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:40.454399 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1240/2460] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:40.458454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feba5c-36f3-4e96-b205-0ea87654c585 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:40.468113 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feba5c-36f3-4e96-b205-0ea87654c585 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:40.473206 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c5726-6455-4434-bce4-43e804bbfaac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:40.473792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c5726-6455-4434-bce4-43e804bbfaac admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:40.474067 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c5726-6455-4434-bce4-43e804bbfaac admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:40.474334 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c5726-6455-4434-bce4-43e804bbfaac admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:40.474966 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21c5726-6455-4434-bce4-43e804bbfaac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:42:40.475260 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21c5726-6455-4434-bce4-43e804bbfaac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:40.476874 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21c5726-6455-4434-bce4-43e804bbfaac admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:40.488751 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1221/2461] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:42:40.492845 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cce053-3608-4c86-8476-3cda0f3f7e29 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:40.502553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cce053-3608-4c86-8476-3cda0f3f7e29 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:40.507249 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc44083-fd32-48b9-8ca1-4cea96021a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:40.507847 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc44083-fd32-48b9-8ca1-4cea96021a25 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:40.508125 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc44083-fd32-48b9-8ca1-4cea96021a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:40.508394 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc44083-fd32-48b9-8ca1-4cea96021a25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:40.509055 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc44083-fd32-48b9-8ca1-4cea96021a25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:40.510969 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc44083-fd32-48b9-8ca1-4cea96021a25 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:40.538286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1241/2462] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:42:40 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:42:40.542342 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c5726-6455-4434-bce4-43e804bbfaac admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:40.548187 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:40.554723 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c5726-6455-4434-bce4-43e804bbfaac admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:40.611415 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5235c-2b07-4a4e-8023-d8fbec16b139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:40.611939 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5235c-2b07-4a4e-8023-d8fbec16b139 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:40.612219 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5235c-2b07-4a4e-8023-d8fbec16b139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:40.612491 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5235c-2b07-4a4e-8023-d8fbec16b139 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:40.613154 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5235c-2b07-4a4e-8023-d8fbec16b139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:40.615044 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5235c-2b07-4a4e-8023-d8fbec16b139 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:40.635687 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1222/2463] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:42:40 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:42:40.639780 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc44083-fd32-48b9-8ca1-4cea96021a25 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:40.650928 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc44083-fd32-48b9-8ca1-4cea96021a25 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:40.655597 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350f6f83-c59a-49e4-81e4-c8476f59a55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:40.656826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f6f83-c59a-49e4-81e4-c8476f59a55a admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:40.657222 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f6f83-c59a-49e4-81e4-c8476f59a55a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:40.657501 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f6f83-c59a-49e4-81e4-c8476f59a55a admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:40.658159 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350f6f83-c59a-49e4-81e4-c8476f59a55a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:40.659737 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350f6f83-c59a-49e4-81e4-c8476f59a55a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:40.669622 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1242/2464] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:42:40.674062 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5235c-2b07-4a4e-8023-d8fbec16b139 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:40.684103 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5235c-2b07-4a4e-8023-d8fbec16b139 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:40.688835 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b81d4-2d63-4a64-b8f9-c6796fd8e0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:40.689430 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b81d4-2d63-4a64-b8f9-c6796fd8e0de admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:40.689706 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b81d4-2d63-4a64-b8f9-c6796fd8e0de admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:40.689974 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b81d4-2d63-4a64-b8f9-c6796fd8e0de admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:40.690608 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b81d4-2d63-4a64-b8f9-c6796fd8e0de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:40.692295 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b81d4-2d63-4a64-b8f9-c6796fd8e0de admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:40.703288 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1223/2465] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:42:40.707372 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350f6f83-c59a-49e4-81e4-c8476f59a55a admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:40.717506 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350f6f83-c59a-49e4-81e4-c8476f59a55a admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:40.723051 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294a35a-81e1-465f-90dd-087b7082aaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:40.723708 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294a35a-81e1-465f-90dd-087b7082aaaa admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:40.724018 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294a35a-81e1-465f-90dd-087b7082aaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:40.724303 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294a35a-81e1-465f-90dd-087b7082aaaa admin admin] PATH_INFO: `/v3/projects/73df73c7c77d435ba5c5725c2eb1567b/users/cbee91e2e9974c02bca084b1777551fd/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:40.745636 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9294a35a-81e1-465f-90dd-087b7082aaaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=73df73c7c77d435ba5c5725c2eb1567b, user_id=cbee91e2e9974c02bca084b1777551fd, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:40.748255 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9294a35a-81e1-465f-90dd-087b7082aaaa admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:40.761828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1243/2466] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:42:40 2022] PUT /identity/v3/projects/73df73c7c77d435ba5c5725c2eb1567b/users/cbee91e2e9974c02bca084b1777551fd/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:42:40.766788 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7c1f5-7327-4b1d-a707-6b24e339fe1d None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:40.767289 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7c1f5-7327-4b1d-a707-6b24e339fe1d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:40.767394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7c1f5-7327-4b1d-a707-6b24e339fe1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:40.786670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-b3b7c1f5-7327-4b1d-a707-6b24e339fe1d None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:40.787858 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b3b7c1f5-7327-4b1d-a707-6b24e339fe1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:40.844819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1224/2467] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:40.854223 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294a35a-81e1-465f-90dd-087b7082aaaa admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:40.855911 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9294a35a-81e1-465f-90dd-087b7082aaaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:40.912615 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294a35a-81e1-465f-90dd-087b7082aaaa admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:40.916232 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337b6e2-0716-4620-b316-adc7fdfbdd95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:40.916869 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337b6e2-0716-4620-b316-adc7fdfbdd95 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:40.917148 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337b6e2-0716-4620-b316-adc7fdfbdd95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:40.917420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337b6e2-0716-4620-b316-adc7fdfbdd95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:40.926762 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337b6e2-0716-4620-b316-adc7fdfbdd95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:40.928187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337b6e2-0716-4620-b316-adc7fdfbdd95 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:40.938042 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1244/2468] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:42:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:42:41.548767 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787d476-da8c-4aa8-bd8e-89e38eeabd25 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:41.549058 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787d476-da8c-4aa8-bd8e-89e38eeabd25 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:41.549337 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787d476-da8c-4aa8-bd8e-89e38eeabd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:41.579493 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-6787d476-da8c-4aa8-bd8e-89e38eeabd25 None None] MFA Rules not processed for user `cbee91e2e9974c02bca084b1777551fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:41.580826 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6787d476-da8c-4aa8-bd8e-89e38eeabd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:41.648374 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1225/2469] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:41.659326 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337b6e2-0716-4620-b316-adc7fdfbdd95 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:41.660794 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d337b6e2-0716-4620-b316-adc7fdfbdd95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:41.670977 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:41.724687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337b6e2-0716-4620-b316-adc7fdfbdd95 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:41.728440 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0d4a1-ab3b-4eb8-becf-af1ba7bc936d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:41.729081 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0d4a1-ab3b-4eb8-becf-af1ba7bc936d service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:41.729357 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0d4a1-ab3b-4eb8-becf-af1ba7bc936d service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:41.729624 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0d4a1-ab3b-4eb8-becf-af1ba7bc936d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:41.739277 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0d4a1-ab3b-4eb8-becf-af1ba7bc936d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:41.740722 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0d4a1-ab3b-4eb8-becf-af1ba7bc936d service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:41.749833 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1245/2470] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:42:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:42:43.103575 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e70909-322b-45ba-afef-285e0ede7bf7 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:43.104073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e70909-322b-45ba-afef-285e0ede7bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:43.104244 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e70909-322b-45ba-afef-285e0ede7bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:43.123570 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-02e70909-322b-45ba-afef-285e0ede7bf7 None None] MFA Rules not processed for user `cbee91e2e9974c02bca084b1777551fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:43.125406 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-02e70909-322b-45ba-afef-285e0ede7bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:43.132409 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1226/2471] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:43.362935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0d4a1-ab3b-4eb8-becf-af1ba7bc936d service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:43.364337 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-57a0d4a1-ab3b-4eb8-becf-af1ba7bc936d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:43.377461 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0d4a1-ab3b-4eb8-becf-af1ba7bc936d service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:43.381027 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e4a40-79fd-4c8c-80b8-3a4f6d6062b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:43.381954 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e4a40-79fd-4c8c-80b8-3a4f6d6062b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:43.382263 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e4a40-79fd-4c8c-80b8-3a4f6d6062b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:43.382536 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e4a40-79fd-4c8c-80b8-3a4f6d6062b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:43.384257 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-303e4a40-79fd-4c8c-80b8-3a4f6d6062b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:43.394263 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e4a40-79fd-4c8c-80b8-3a4f6d6062b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:43.395681 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e4a40-79fd-4c8c-80b8-3a4f6d6062b8 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:43.403901 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1246/2472] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:42:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:42:43.715738 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e70909-322b-45ba-afef-285e0ede7bf7 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:43.717151 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-02e70909-322b-45ba-afef-285e0ede7bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:43.723628 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:43.789172 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e70909-322b-45ba-afef-285e0ede7bf7 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:43.794044 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199e0bb-bae2-4bc0-a13d-c1e19454f14e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfbaaf32475a4c82a34948602d8f4e35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:43.794645 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199e0bb-bae2-4bc0-a13d-c1e19454f14e service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:43.794923 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199e0bb-bae2-4bc0-a13d-c1e19454f14e service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:43.795195 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199e0bb-bae2-4bc0-a13d-c1e19454f14e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:43.797494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e199e0bb-bae2-4bc0-a13d-c1e19454f14e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:43.809116 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199e0bb-bae2-4bc0-a13d-c1e19454f14e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:43.810666 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199e0bb-bae2-4bc0-a13d-c1e19454f14e service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:43.820319 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1227/2473] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:42:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:42:45.503138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e4a40-79fd-4c8c-80b8-3a4f6d6062b8 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:45.512152 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e4a40-79fd-4c8c-80b8-3a4f6d6062b8 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:45.516374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80d5ff-1c56-4222-972a-94bf584bf1d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:45.516995 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80d5ff-1c56-4222-972a-94bf584bf1d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:45.517275 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80d5ff-1c56-4222-972a-94bf584bf1d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:45.517547 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80d5ff-1c56-4222-972a-94bf584bf1d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:45.519259 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7b80d5ff-1c56-4222-972a-94bf584bf1d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:45.528899 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80d5ff-1c56-4222-972a-94bf584bf1d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:45.530317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80d5ff-1c56-4222-972a-94bf584bf1d9 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:45.539129 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1247/2474] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:42:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:42:51.380827 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199e0bb-bae2-4bc0-a13d-c1e19454f14e service placement] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:51.382317 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e199e0bb-bae2-4bc0-a13d-c1e19454f14e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:51.462632 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199e0bb-bae2-4bc0-a13d-c1e19454f14e service placement] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:51.468189 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef92bb9-64c1-45ea-a82c-718046ee3de1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:51.468821 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef92bb9-64c1-45ea-a82c-718046ee3de1 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:51.469104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef92bb9-64c1-45ea-a82c-718046ee3de1 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:51.469486 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef92bb9-64c1-45ea-a82c-718046ee3de1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:51.471213 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-aef92bb9-64c1-45ea-a82c-718046ee3de1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:51.483194 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef92bb9-64c1-45ea-a82c-718046ee3de1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:51.485271 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef92bb9-64c1-45ea-a82c-718046ee3de1 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:51.495757 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1228/2475] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:42:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:42:55.535322 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=35 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:42:58.330961 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a41c0-6b37-438e-b449-17aff6aba69d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:58.331517 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a41c0-6b37-438e-b449-17aff6aba69d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:58.331517 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a41c0-6b37-438e-b449-17aff6aba69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:58.351067 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-e51a41c0-6b37-438e-b449-17aff6aba69d None None] MFA Rules not processed for user `cbee91e2e9974c02bca084b1777551fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:42:58.352246 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e51a41c0-6b37-438e-b449-17aff6aba69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:58.361883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1248/2476] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:42:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:42:58.588953 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef92bb9-64c1-45ea-a82c-718046ee3de1 service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:42:58.599028 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef92bb9-64c1-45ea-a82c-718046ee3de1 service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:42:58.602651 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55545be-7bb6-4f25-845c-40905194657e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:42:58.603244 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55545be-7bb6-4f25-845c-40905194657e service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:42:58.603524 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55545be-7bb6-4f25-845c-40905194657e service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:42:58.603796 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55545be-7bb6-4f25-845c-40905194657e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:42:58.605554 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a55545be-7bb6-4f25-845c-40905194657e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:42:58.615441 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55545be-7bb6-4f25-845c-40905194657e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:42:58.616887 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55545be-7bb6-4f25-845c-40905194657e service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:42:58.624721 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1229/2477] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:42:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:43:08.346285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:43:08.621138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=70 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:43:18.453428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8672fb-20d2-41a7-82a9-2c7dd164dd3b None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:43:18.453901 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8672fb-20d2-41a7-82a9-2c7dd164dd3b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:43:18.454003 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8672fb-20d2-41a7-82a9-2c7dd164dd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:43:18.472804 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-1b8672fb-20d2-41a7-82a9-2c7dd164dd3b None None] MFA Rules not processed for user `7a59f547a20e48ee88e3e620ba1e87f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:43:18.473986 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1b8672fb-20d2-41a7-82a9-2c7dd164dd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:43:18.528953 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1249/2478] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Oct 20 06:43:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:43:18.575229 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55545be-7bb6-4f25-845c-40905194657e service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:43:18.576718 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a55545be-7bb6-4f25-845c-40905194657e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:43:18.648976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55545be-7bb6-4f25-845c-40905194657e service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:43:18.654004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c22e31f-646e-441b-9414-7c0be2deb4d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeeab506ab5c4d6386fc899086f0abf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:43:18.654606 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c22e31f-646e-441b-9414-7c0be2deb4d4 service swift] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:43:18.654884 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c22e31f-646e-441b-9414-7c0be2deb4d4 service swift] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:43:18.655158 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c22e31f-646e-441b-9414-7c0be2deb4d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:43:18.663992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c22e31f-646e-441b-9414-7c0be2deb4d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:43:18.665602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c22e31f-646e-441b-9414-7c0be2deb4d4 service swift] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:43:18.674141 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1230/2479] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Oct 20 06:43:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 06:43:28.522446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=25 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:43:28.670392 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=32 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:43:33.792807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b595244-e6ba-460d-91df-cb97a925a84c None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:43:33.793270 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b595244-e6ba-460d-91df-cb97a925a84c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:43:33.793359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b595244-e6ba-460d-91df-cb97a925a84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:43:33.812032 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-8b595244-e6ba-460d-91df-cb97a925a84c None None] MFA Rules not processed for user `7a59f547a20e48ee88e3e620ba1e87f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:43:33.813916 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8b595244-e6ba-460d-91df-cb97a925a84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:43:33.821139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1250/2480] 10.222.0.20 () {58 vars in 967 bytes} [Thu Oct 20 06:43:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:43:33.834446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c22e31f-646e-441b-9414-7c0be2deb4d4 service swift] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:43:33.845280 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c22e31f-646e-441b-9414-7c0be2deb4d4 service swift] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:43:33.850204 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3f93f-740f-447f-9236-75854c925d72 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeeab506ab5c4d6386fc899086f0abf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:43:33.850787 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3f93f-740f-447f-9236-75854c925d72 service swift] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:43:33.851085 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3f93f-740f-447f-9236-75854c925d72 service swift] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:43:33.851359 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3f93f-740f-447f-9236-75854c925d72 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:43:33.859401 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3f93f-740f-447f-9236-75854c925d72 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:43:33.861220 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3f93f-740f-447f-9236-75854c925d72 service swift] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:43:33.869393 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1231/2481] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Oct 20 06:43:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 06:43:35.955904 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b595244-e6ba-460d-91df-cb97a925a84c None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:43:35.957425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8b595244-e6ba-460d-91df-cb97a925a84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:43:35.972499 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b595244-e6ba-460d-91df-cb97a925a84c None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:43:35.978134 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8068f-145c-471f-97ab-8eb8f9b5ff6f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfbaaf32475a4c82a34948602d8f4e35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:43:35.978754 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8068f-145c-471f-97ab-8eb8f9b5ff6f service placement] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:43:35.979035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8068f-145c-471f-97ab-8eb8f9b5ff6f service placement] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:43:35.979567 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8068f-145c-471f-97ab-8eb8f9b5ff6f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:43:35.981067 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a4a8068f-145c-471f-97ab-8eb8f9b5ff6f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:43:35.996350 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a8068f-145c-471f-97ab-8eb8f9b5ff6f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:43:35.997998 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a8068f-145c-471f-97ab-8eb8f9b5ff6f service placement] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:43:36.007857 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3f93f-740f-447f-9236-75854c925d72 service swift] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:43:36.023391 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1251/2482] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:43:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:43:36.023970 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3f93f-740f-447f-9236-75854c925d72 service swift] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:43:36.030503 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93107d-0138-435d-b354-96c8b9ed355b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:43:36.031501 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93107d-0138-435d-b354-96c8b9ed355b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:43:36.031825 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93107d-0138-435d-b354-96c8b9ed355b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:43:36.032326 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93107d-0138-435d-b354-96c8b9ed355b admin admin] PATH_INFO: `/v3/users/cbee91e2e9974c02bca084b1777551fd` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:43:36.033762 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93107d-0138-435d-b354-96c8b9ed355b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbee91e2e9974c02bca084b1777551fd)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:43:36.036020 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93107d-0138-435d-b354-96c8b9ed355b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:43:36.097131 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-dd93107d-0138-435d-b354-96c8b9ed355b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbee91e2e9974c02bca084b1777551fd'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:43:36.097802 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-dd93107d-0138-435d-b354-96c8b9ed355b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbee91e2e9974c02bca084b1777551fd'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:43:36.104879 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-dd93107d-0138-435d-b354-96c8b9ed355b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbee91e2e9974c02bca084b1777551fd'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:43:36.117376 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-dd93107d-0138-435d-b354-96c8b9ed355b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbee91e2e9974c02bca084b1777551fd'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:43:36.124144 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-dd93107d-0138-435d-b354-96c8b9ed355b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbee91e2e9974c02bca084b1777551fd'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:43:36.130366 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1232/2483] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:43:36 2022] DELETE /identity/v3/users/cbee91e2e9974c02bca084b1777551fd => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:43:36.297434 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8068f-145c-471f-97ab-8eb8f9b5ff6f service placement] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:43:36.317980 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8068f-145c-471f-97ab-8eb8f9b5ff6f service placement] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:43:36.373222 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6f00c-e397-49b3-b6f8-074e265760c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:43:36.373842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6f00c-e397-49b3-b6f8-074e265760c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:43:36.374118 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6f00c-e397-49b3-b6f8-074e265760c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:43:36.374387 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6f00c-e397-49b3-b6f8-074e265760c2 admin admin] PATH_INFO: `/v3/projects/73df73c7c77d435ba5c5725c2eb1567b` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:43:36.375473 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6f00c-e397-49b3-b6f8-074e265760c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73df73c7c77d435ba5c5725c2eb1567b)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:43:36.378373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6f00c-e397-49b3-b6f8-074e265760c2 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:43:36.423859 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-09b6f00c-e397-49b3-b6f8-074e265760c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73df73c7c77d435ba5c5725c2eb1567b', 'request_id': 'req-09b6f00c-e397-49b3-b6f8-074e265760c2'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:43:36.429365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-09b6f00c-e397-49b3-b6f8-074e265760c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73df73c7c77d435ba5c5725c2eb1567b', 'request_id': 'req-09b6f00c-e397-49b3-b6f8-074e265760c2'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:43:36.435440 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-09b6f00c-e397-49b3-b6f8-074e265760c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73df73c7c77d435ba5c5725c2eb1567b', 'request_id': 'req-09b6f00c-e397-49b3-b6f8-074e265760c2'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:43:36.448046 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-09b6f00c-e397-49b3-b6f8-074e265760c2 admin admin] The token cache is being invalidate because project 73df73c7c77d435ba5c5725c2eb1567b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:43:36.448332 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-09b6f00c-e397-49b3-b6f8-074e265760c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:43:36.448969 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-09b6f00c-e397-49b3-b6f8-074e265760c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:43:36.450415 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1252/2484] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:43:36 2022] DELETE /identity/v3/projects/73df73c7c77d435ba5c5725c2eb1567b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:43:36.455317 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf4625-206e-4c35-8f76-d51df2fd3375 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:43:36.455575 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf4625-206e-4c35-8f76-d51df2fd3375 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:43:36.455898 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf4625-206e-4c35-8f76-d51df2fd3375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:43:36.475405 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-eaaf4625-206e-4c35-8f76-d51df2fd3375 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:43:36.476526 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-eaaf4625-206e-4c35-8f76-d51df2fd3375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:43:36.534958 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1233/2485] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:43:36.542426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319b5b4-1acf-47f5-9d4f-2b18cb1e3d09 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:43:36.542710 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319b5b4-1acf-47f5-9d4f-2b18cb1e3d09 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:43:36.542982 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319b5b4-1acf-47f5-9d4f-2b18cb1e3d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:43:36.562378 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-f319b5b4-1acf-47f5-9d4f-2b18cb1e3d09 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:43:36.563725 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f319b5b4-1acf-47f5-9d4f-2b18cb1e3d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:43:36.571451 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1253/2486] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:43:36.575531 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf4625-206e-4c35-8f76-d51df2fd3375 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:43:36.585815 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf4625-206e-4c35-8f76-d51df2fd3375 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:43:36.590516 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177ab7e-1700-4a2e-98ba-a1d90fa333c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:43:36.591141 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177ab7e-1700-4a2e-98ba-a1d90fa333c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:43:36.591422 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177ab7e-1700-4a2e-98ba-a1d90fa333c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:43:36.591695 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177ab7e-1700-4a2e-98ba-a1d90fa333c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:43:36.592333 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177ab7e-1700-4a2e-98ba-a1d90fa333c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:43:36.592639 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177ab7e-1700-4a2e-98ba-a1d90fa333c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:43:36.594257 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177ab7e-1700-4a2e-98ba-a1d90fa333c2 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:43:36.606242 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1234/2487] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:43:36.610338 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319b5b4-1acf-47f5-9d4f-2b18cb1e3d09 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:43:36.619283 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319b5b4-1acf-47f5-9d4f-2b18cb1e3d09 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:43:36.623835 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c00f96-5cc2-431c-b382-434406117109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:43:36.624412 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c00f96-5cc2-431c-b382-434406117109 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:43:36.624686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c00f96-5cc2-431c-b382-434406117109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:43:36.624980 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c00f96-5cc2-431c-b382-434406117109 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:43:36.625597 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c00f96-5cc2-431c-b382-434406117109 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:43:36.628326 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c00f96-5cc2-431c-b382-434406117109 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:43:36.661017 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1254/2488] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:43:36 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:43:36.665228 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177ab7e-1700-4a2e-98ba-a1d90fa333c2 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:43:36.675304 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177ab7e-1700-4a2e-98ba-a1d90fa333c2 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:43:36.718901 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:43:36.732776 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3173c-1090-43bb-8c4f-915b391bd2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:43:36.733378 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3173c-1090-43bb-8c4f-915b391bd2ec admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:43:36.733655 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3173c-1090-43bb-8c4f-915b391bd2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:43:36.733923 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3173c-1090-43bb-8c4f-915b391bd2ec admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:43:36.734552 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b3173c-1090-43bb-8c4f-915b391bd2ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:43:36.736451 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b3173c-1090-43bb-8c4f-915b391bd2ec admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:43:36.757186 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1235/2489] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:43:36 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:43:36.761217 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c00f96-5cc2-431c-b382-434406117109 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:43:36.771773 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c00f96-5cc2-431c-b382-434406117109 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:43:36.776198 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee058623-c9de-4e12-8f00-eafaaddefdeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:43:36.776807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee058623-c9de-4e12-8f00-eafaaddefdeb admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:43:36.777083 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee058623-c9de-4e12-8f00-eafaaddefdeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:43:36.777349 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee058623-c9de-4e12-8f00-eafaaddefdeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:43:36.777977 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee058623-c9de-4e12-8f00-eafaaddefdeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:43:36.779540 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee058623-c9de-4e12-8f00-eafaaddefdeb admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:43:36.789340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1255/2490] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:43:36.793126 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3173c-1090-43bb-8c4f-915b391bd2ec admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:43:36.802906 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3173c-1090-43bb-8c4f-915b391bd2ec admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:43:36.807440 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8673186-9965-4a9d-bf35-7222c5abe368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:43:36.808024 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8673186-9965-4a9d-bf35-7222c5abe368 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:43:36.808303 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8673186-9965-4a9d-bf35-7222c5abe368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:43:36.808573 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8673186-9965-4a9d-bf35-7222c5abe368 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:43:36.809232 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8673186-9965-4a9d-bf35-7222c5abe368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:43:36.810806 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8673186-9965-4a9d-bf35-7222c5abe368 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:43:36.821578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1236/2491] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:43:36.825725 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee058623-c9de-4e12-8f00-eafaaddefdeb admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:43:36.835517 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee058623-c9de-4e12-8f00-eafaaddefdeb admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:43:36.840023 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083db4a5-addb-4870-98cf-3ead40c51ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:43:36.840656 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083db4a5-addb-4870-98cf-3ead40c51ac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:43:36.840963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083db4a5-addb-4870-98cf-3ead40c51ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:43:36.841234 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083db4a5-addb-4870-98cf-3ead40c51ac2 admin admin] PATH_INFO: `/v3/projects/e746117abe724498b704fd0ee617f761/users/5570fb1f091849b0b8013678cb3812cf/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:43:36.855905 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083db4a5-addb-4870-98cf-3ead40c51ac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e746117abe724498b704fd0ee617f761, user_id=5570fb1f091849b0b8013678cb3812cf, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:43:36.858540 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083db4a5-addb-4870-98cf-3ead40c51ac2 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:43:36.871799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1256/2492] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:43:36 2022] PUT /identity/v3/projects/e746117abe724498b704fd0ee617f761/users/5570fb1f091849b0b8013678cb3812cf/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:43:36.882032 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccae144-08ce-4b56-99d8-68da72292bb3 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:43:36.882348 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccae144-08ce-4b56-99d8-68da72292bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:43:36.882623 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccae144-08ce-4b56-99d8-68da72292bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:43:36.910496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-bccae144-08ce-4b56-99d8-68da72292bb3 None None] MFA Rules not processed for user `5570fb1f091849b0b8013678cb3812cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:43:36.911696 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-bccae144-08ce-4b56-99d8-68da72292bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:43:36.973744 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1237/2493] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:43:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:43:36.985925 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083db4a5-addb-4870-98cf-3ead40c51ac2 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:43:36.987686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-083db4a5-addb-4870-98cf-3ead40c51ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:43:36.997341 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:43:37.048980 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083db4a5-addb-4870-98cf-3ead40c51ac2 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:43:37.052588 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951a679-86b2-4e3a-a649-d4ea908656a8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dcfe5244bd04c00b6dde1d6f795d4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:43:37.053476 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951a679-86b2-4e3a-a649-d4ea908656a8 service glance] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:43:37.053581 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951a679-86b2-4e3a-a649-d4ea908656a8 service glance] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:43:37.053814 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951a679-86b2-4e3a-a649-d4ea908656a8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:43:37.062572 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951a679-86b2-4e3a-a649-d4ea908656a8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:43:37.063996 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951a679-86b2-4e3a-a649-d4ea908656a8 service glance] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:43:37.072967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1257/2494] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:43:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:43:46.971111 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:43:47.068783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=23 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:44:09.342868 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccae144-08ce-4b56-99d8-68da72292bb3 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:09.423977 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccae144-08ce-4b56-99d8-68da72292bb3 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:09.429158 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed3472-7699-4e09-8df8-cded6b7f3322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:09.429785 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed3472-7699-4e09-8df8-cded6b7f3322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:09.430066 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed3472-7699-4e09-8df8-cded6b7f3322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:09.430340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed3472-7699-4e09-8df8-cded6b7f3322 admin admin] PATH_INFO: `/v3/users/5570fb1f091849b0b8013678cb3812cf` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:09.431465 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efed3472-7699-4e09-8df8-cded6b7f3322 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5570fb1f091849b0b8013678cb3812cf)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:09.433413 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efed3472-7699-4e09-8df8-cded6b7f3322 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:09.478589 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-efed3472-7699-4e09-8df8-cded6b7f3322 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5570fb1f091849b0b8013678cb3812cf'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:09.479225 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-efed3472-7699-4e09-8df8-cded6b7f3322 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5570fb1f091849b0b8013678cb3812cf'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:09.486942 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-efed3472-7699-4e09-8df8-cded6b7f3322 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5570fb1f091849b0b8013678cb3812cf'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:09.499464 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-efed3472-7699-4e09-8df8-cded6b7f3322 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5570fb1f091849b0b8013678cb3812cf'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:09.507168 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-efed3472-7699-4e09-8df8-cded6b7f3322 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5570fb1f091849b0b8013678cb3812cf'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:09.512982 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1238/2495] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:44:09 2022] DELETE /identity/v3/users/5570fb1f091849b0b8013678cb3812cf => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:44:09.517551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18e486-e297-44be-8bd2-23ce4066be7e None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:09.518024 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18e486-e297-44be-8bd2-23ce4066be7e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:09.518126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18e486-e297-44be-8bd2-23ce4066be7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:09.547204 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-4b18e486-e297-44be-8bd2-23ce4066be7e None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:44:09.548393 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4b18e486-e297-44be-8bd2-23ce4066be7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:09.606114 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1258/2496] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:44:09.615649 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed3472-7699-4e09-8df8-cded6b7f3322 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:09.617108 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-efed3472-7699-4e09-8df8-cded6b7f3322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:09.682829 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed3472-7699-4e09-8df8-cded6b7f3322 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:09.686506 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c671a-ad29-4741-a029-a34172cbadb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:09.687103 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c671a-ad29-4741-a029-a34172cbadb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:09.687386 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c671a-ad29-4741-a029-a34172cbadb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:09.687660 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c671a-ad29-4741-a029-a34172cbadb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:09.694017 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:44:09.698811 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c671a-ad29-4741-a029-a34172cbadb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:09.700559 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c671a-ad29-4741-a029-a34172cbadb2 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:09.711596 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1239/2497] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:44:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:44:09.986595 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18e486-e297-44be-8bd2-23ce4066be7e None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:09.988049 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4b18e486-e297-44be-8bd2-23ce4066be7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:10.001757 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18e486-e297-44be-8bd2-23ce4066be7e None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:10.006525 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0e35a-2854-49c1-81dd-16d14a65bc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:10.007185 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0e35a-2854-49c1-81dd-16d14a65bc3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:10.008002 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0e35a-2854-49c1-81dd-16d14a65bc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:10.008303 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0e35a-2854-49c1-81dd-16d14a65bc3f admin admin] PATH_INFO: `/v3/projects/e746117abe724498b704fd0ee617f761` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:10.009558 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0e35a-2854-49c1-81dd-16d14a65bc3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e746117abe724498b704fd0ee617f761)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:10.011468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0e35a-2854-49c1-81dd-16d14a65bc3f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:10.061324 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9ec0e35a-2854-49c1-81dd-16d14a65bc3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e746117abe724498b704fd0ee617f761', 'request_id': 'req-9ec0e35a-2854-49c1-81dd-16d14a65bc3f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:10.067084 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9ec0e35a-2854-49c1-81dd-16d14a65bc3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e746117abe724498b704fd0ee617f761', 'request_id': 'req-9ec0e35a-2854-49c1-81dd-16d14a65bc3f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:10.073395 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9ec0e35a-2854-49c1-81dd-16d14a65bc3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e746117abe724498b704fd0ee617f761', 'request_id': 'req-9ec0e35a-2854-49c1-81dd-16d14a65bc3f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:10.086452 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9ec0e35a-2854-49c1-81dd-16d14a65bc3f admin admin] The token cache is being invalidate because project e746117abe724498b704fd0ee617f761 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:44:10.086734 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9ec0e35a-2854-49c1-81dd-16d14a65bc3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:10.087351 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9ec0e35a-2854-49c1-81dd-16d14a65bc3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:10.088852 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1259/2498] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:44:09 2022] DELETE /identity/v3/projects/e746117abe724498b704fd0ee617f761 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:44:10.324340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c671a-ad29-4741-a029-a34172cbadb2 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:10.325837 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c07c671a-ad29-4741-a029-a34172cbadb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:10.417200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c671a-ad29-4741-a029-a34172cbadb2 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:10.475084 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbb257-886b-4ba4-807c-4e586be3585d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfbaaf32475a4c82a34948602d8f4e35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:10.475687 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbb257-886b-4ba4-807c-4e586be3585d service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:10.475968 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbb257-886b-4ba4-807c-4e586be3585d service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:10.476242 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbb257-886b-4ba4-807c-4e586be3585d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:10.477980 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-64fbb257-886b-4ba4-807c-4e586be3585d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:10.489984 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fbb257-886b-4ba4-807c-4e586be3585d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:10.491569 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fbb257-886b-4ba4-807c-4e586be3585d service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:10.503750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1240/2499] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:44:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:44:14.245514 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0e35a-2854-49c1-81dd-16d14a65bc3f admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:14.246947 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9ec0e35a-2854-49c1-81dd-16d14a65bc3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:14.313322 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0e35a-2854-49c1-81dd-16d14a65bc3f admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:14.317167 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd8a6c-b2a7-48c4-886d-327fcc7f5358 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:14.317775 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd8a6c-b2a7-48c4-886d-327fcc7f5358 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:14.318053 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd8a6c-b2a7-48c4-886d-327fcc7f5358 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:14.318325 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd8a6c-b2a7-48c4-886d-327fcc7f5358 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:14.320032 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a9bd8a6c-b2a7-48c4-886d-327fcc7f5358 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:14.338355 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:44:14.383396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd8a6c-b2a7-48c4-886d-327fcc7f5358 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:14.384860 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd8a6c-b2a7-48c4-886d-327fcc7f5358 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:14.394978 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1260/2500] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:44:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:44:17.513263 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbb257-886b-4ba4-807c-4e586be3585d service placement] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:17.514763 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-64fbb257-886b-4ba4-807c-4e586be3585d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:17.533768 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbb257-886b-4ba4-807c-4e586be3585d service placement] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:17.539367 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7df99-f74f-4935-a16f-eb20a92823f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:17.540195 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7df99-f74f-4935-a16f-eb20a92823f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:17.540476 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7df99-f74f-4935-a16f-eb20a92823f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:17.547206 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7df99-f74f-4935-a16f-eb20a92823f0 admin admin] PATH_INFO: `/v3/users/2c6be23065c643939055a3de375187d5` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:17.548605 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7df99-f74f-4935-a16f-eb20a92823f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c6be23065c643939055a3de375187d5)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:17.550585 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7df99-f74f-4935-a16f-eb20a92823f0 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:17.596880 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-f7e7df99-f74f-4935-a16f-eb20a92823f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c6be23065c643939055a3de375187d5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:17.598150 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-f7e7df99-f74f-4935-a16f-eb20a92823f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c6be23065c643939055a3de375187d5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:17.605040 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-f7e7df99-f74f-4935-a16f-eb20a92823f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c6be23065c643939055a3de375187d5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:17.606942 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:44:17.619243 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-f7e7df99-f74f-4935-a16f-eb20a92823f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c6be23065c643939055a3de375187d5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:17.626368 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-f7e7df99-f74f-4935-a16f-eb20a92823f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c6be23065c643939055a3de375187d5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:17.632484 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1241/2501] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:44:17 2022] DELETE /identity/v3/users/2c6be23065c643939055a3de375187d5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:44:17.809048 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd8a6c-b2a7-48c4-886d-327fcc7f5358 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:17.819365 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd8a6c-b2a7-48c4-886d-327fcc7f5358 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:17.885430 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba7c6fc-a8ee-4249-9ce9-1694ad32913f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:17.886076 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7c6fc-a8ee-4249-9ce9-1694ad32913f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:17.886354 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7c6fc-a8ee-4249-9ce9-1694ad32913f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:17.886782 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7c6fc-a8ee-4249-9ce9-1694ad32913f admin admin] PATH_INFO: `/v3/projects/b90087118fe040389f291d846461c0a7` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:17.887736 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba7c6fc-a8ee-4249-9ce9-1694ad32913f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b90087118fe040389f291d846461c0a7)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:17.889685 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba7c6fc-a8ee-4249-9ce9-1694ad32913f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:17.938982 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-bba7c6fc-a8ee-4249-9ce9-1694ad32913f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b90087118fe040389f291d846461c0a7', 'request_id': 'req-bba7c6fc-a8ee-4249-9ce9-1694ad32913f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:17.944787 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-bba7c6fc-a8ee-4249-9ce9-1694ad32913f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b90087118fe040389f291d846461c0a7', 'request_id': 'req-bba7c6fc-a8ee-4249-9ce9-1694ad32913f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:17.950955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-bba7c6fc-a8ee-4249-9ce9-1694ad32913f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b90087118fe040389f291d846461c0a7', 'request_id': 'req-bba7c6fc-a8ee-4249-9ce9-1694ad32913f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:17.963182 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-bba7c6fc-a8ee-4249-9ce9-1694ad32913f admin admin] The token cache is being invalidate because project b90087118fe040389f291d846461c0a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:44:17.963472 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-bba7c6fc-a8ee-4249-9ce9-1694ad32913f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:17.964094 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-bba7c6fc-a8ee-4249-9ce9-1694ad32913f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:17.965599 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1261/2502] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:44:17 2022] DELETE /identity/v3/projects/b90087118fe040389f291d846461c0a7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:44:17.971271 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e297f-e3f7-4bd8-8cc4-c635382f2975 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:17.971558 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e297f-e3f7-4bd8-8cc4-c635382f2975 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:17.971836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e297f-e3f7-4bd8-8cc4-c635382f2975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:17.992856 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c22e297f-e3f7-4bd8-8cc4-c635382f2975 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:44:17.994072 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c22e297f-e3f7-4bd8-8cc4-c635382f2975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:18.053980 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1242/2503] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:44:18.061645 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b50fa-0605-433f-b0e2-e7fc9d539599 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:18.061929 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b50fa-0605-433f-b0e2-e7fc9d539599 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:18.062199 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b50fa-0605-433f-b0e2-e7fc9d539599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:18.082625 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-318b50fa-0605-433f-b0e2-e7fc9d539599 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:44:18.083812 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-318b50fa-0605-433f-b0e2-e7fc9d539599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:18.091521 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1262/2504] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:44:18.095596 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e297f-e3f7-4bd8-8cc4-c635382f2975 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:18.105055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e297f-e3f7-4bd8-8cc4-c635382f2975 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:18.109566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987fc01-5e90-466f-a582-74b78ef14643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:18.110186 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987fc01-5e90-466f-a582-74b78ef14643 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:18.110465 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987fc01-5e90-466f-a582-74b78ef14643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:18.110733 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987fc01-5e90-466f-a582-74b78ef14643 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:18.111367 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e987fc01-5e90-466f-a582-74b78ef14643 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:44:18.111663 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e987fc01-5e90-466f-a582-74b78ef14643 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:18.113449 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e987fc01-5e90-466f-a582-74b78ef14643 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:18.124631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1243/2505] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:44:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:44:18.129775 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b50fa-0605-433f-b0e2-e7fc9d539599 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:18.138890 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b50fa-0605-433f-b0e2-e7fc9d539599 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:18.143597 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0a14a-3d8e-4b7a-a137-b33137bdb2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:18.144209 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0a14a-3d8e-4b7a-a137-b33137bdb2a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:18.145134 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0a14a-3d8e-4b7a-a137-b33137bdb2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:18.145487 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0a14a-3d8e-4b7a-a137-b33137bdb2a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:18.146106 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0a14a-3d8e-4b7a-a137-b33137bdb2a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:18.147992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0a14a-3d8e-4b7a-a137-b33137bdb2a5 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:18.173987 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1263/2506] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:44:18 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:44:18.178671 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987fc01-5e90-466f-a582-74b78ef14643 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:18.188290 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987fc01-5e90-466f-a582-74b78ef14643 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:18.244434 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ed5cd-3601-408d-be12-8e7de413ae10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:18.245060 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ed5cd-3601-408d-be12-8e7de413ae10 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:18.245340 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ed5cd-3601-408d-be12-8e7de413ae10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:18.245606 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ed5cd-3601-408d-be12-8e7de413ae10 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:18.246227 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ed5cd-3601-408d-be12-8e7de413ae10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:18.248116 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ed5cd-3601-408d-be12-8e7de413ae10 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:18.268290 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1244/2507] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:44:18 2022] POST /identity/v3/users => generated 283 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:44:18.272880 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0a14a-3d8e-4b7a-a137-b33137bdb2a5 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:18.282874 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0a14a-3d8e-4b7a-a137-b33137bdb2a5 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:18.287412 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722885d-a947-4ceb-82e0-099eb9d9b1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:18.287995 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722885d-a947-4ceb-82e0-099eb9d9b1ca admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:18.288270 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722885d-a947-4ceb-82e0-099eb9d9b1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:18.288539 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722885d-a947-4ceb-82e0-099eb9d9b1ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:18.289203 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722885d-a947-4ceb-82e0-099eb9d9b1ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:18.290769 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722885d-a947-4ceb-82e0-099eb9d9b1ca admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:18.301284 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1264/2508] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:44:18.305185 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ed5cd-3601-408d-be12-8e7de413ae10 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:18.315187 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ed5cd-3601-408d-be12-8e7de413ae10 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:18.320572 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd4acc-01ab-4ae2-b383-d3236ad45d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:18.321205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd4acc-01ab-4ae2-b383-d3236ad45d01 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:18.321485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd4acc-01ab-4ae2-b383-d3236ad45d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:18.321757 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd4acc-01ab-4ae2-b383-d3236ad45d01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:18.322387 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd4acc-01ab-4ae2-b383-d3236ad45d01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:18.323977 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd4acc-01ab-4ae2-b383-d3236ad45d01 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:18.334718 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1245/2509] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:44:18.338744 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722885d-a947-4ceb-82e0-099eb9d9b1ca admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:18.345228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:44:18.349103 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722885d-a947-4ceb-82e0-099eb9d9b1ca admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:18.355294 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385c11e-d2cc-4917-8753-3c0de0d4da0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:18.356187 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385c11e-d2cc-4917-8753-3c0de0d4da0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:18.356519 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385c11e-d2cc-4917-8753-3c0de0d4da0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:18.356974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385c11e-d2cc-4917-8753-3c0de0d4da0a admin admin] PATH_INFO: `/v3/projects/bf0c9a48152a4ba2b265df31ec724ac5/users/47815b526b0e4f42868fa7e6d03ec098/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:18.374599 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385c11e-d2cc-4917-8753-3c0de0d4da0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf0c9a48152a4ba2b265df31ec724ac5, user_id=47815b526b0e4f42868fa7e6d03ec098, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:18.377469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385c11e-d2cc-4917-8753-3c0de0d4da0a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:18.390968 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1265/2510] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:44:18 2022] PUT /identity/v3/projects/bf0c9a48152a4ba2b265df31ec724ac5/users/47815b526b0e4f42868fa7e6d03ec098/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:44:18.401231 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d37f6-f1be-491e-baaa-00544f1f05e6 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:18.401518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d37f6-f1be-491e-baaa-00544f1f05e6 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:18.401819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d37f6-f1be-491e-baaa-00544f1f05e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:18.429468 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-453d37f6-f1be-491e-baaa-00544f1f05e6 None None] MFA Rules not processed for user `47815b526b0e4f42868fa7e6d03ec098`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:44:18.430702 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-453d37f6-f1be-491e-baaa-00544f1f05e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:18.459992 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:44:18.496931 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1246/2511] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:44:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:44:18.501614 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385c11e-d2cc-4917-8753-3c0de0d4da0a admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:18.561725 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385c11e-d2cc-4917-8753-3c0de0d4da0a admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:18.566437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8cfc8-5f38-4b5b-ae05-94c6e7bb9912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:18.567029 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8cfc8-5f38-4b5b-ae05-94c6e7bb9912 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:18.567303 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8cfc8-5f38-4b5b-ae05-94c6e7bb9912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:18.567571 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8cfc8-5f38-4b5b-ae05-94c6e7bb9912 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:18.568185 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b8cfc8-5f38-4b5b-ae05-94c6e7bb9912 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:18.570117 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b8cfc8-5f38-4b5b-ae05-94c6e7bb9912 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:18.595953 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1266/2512] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:44:18 2022] POST /identity/v3/projects => generated 365 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:44:18.600646 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453d37f6-f1be-491e-baaa-00544f1f05e6 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:18.611097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453d37f6-f1be-491e-baaa-00544f1f05e6 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:18.669145 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff054c75-4559-4402-b450-a3147a43da7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:18.669669 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff054c75-4559-4402-b450-a3147a43da7f admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:18.669945 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff054c75-4559-4402-b450-a3147a43da7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:18.670216 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff054c75-4559-4402-b450-a3147a43da7f admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:18.670836 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff054c75-4559-4402-b450-a3147a43da7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:18.672757 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff054c75-4559-4402-b450-a3147a43da7f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:18.694224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1247/2513] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:44:18 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:44:18.698175 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8cfc8-5f38-4b5b-ae05-94c6e7bb9912 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:18.708334 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8cfc8-5f38-4b5b-ae05-94c6e7bb9912 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:18.713060 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1ce21-8732-4104-8ed5-69e6780b1fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:18.713654 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1ce21-8732-4104-8ed5-69e6780b1fab admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:18.713935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1ce21-8732-4104-8ed5-69e6780b1fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:18.714211 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1ce21-8732-4104-8ed5-69e6780b1fab admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:18.714851 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1ce21-8732-4104-8ed5-69e6780b1fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:18.716444 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1ce21-8732-4104-8ed5-69e6780b1fab admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:18.728413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1267/2514] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:44:18.732631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff054c75-4559-4402-b450-a3147a43da7f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:18.743346 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff054c75-4559-4402-b450-a3147a43da7f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:18.747922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99de41-6942-445a-82ac-8b56507858ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:18.748561 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99de41-6942-445a-82ac-8b56507858ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:18.748871 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99de41-6942-445a-82ac-8b56507858ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:18.749146 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99de41-6942-445a-82ac-8b56507858ea admin admin] PATH_INFO: `/v3/projects/d2bbb067e3dc42c9ac05e57e459ff2bf/users/4217c49e92714b09a7f76ec096d97e31/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:18.763825 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99de41-6942-445a-82ac-8b56507858ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2bbb067e3dc42c9ac05e57e459ff2bf, user_id=4217c49e92714b09a7f76ec096d97e31, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:18.766498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99de41-6942-445a-82ac-8b56507858ea admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:18.779329 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1248/2515] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:44:18 2022] PUT /identity/v3/projects/d2bbb067e3dc42c9ac05e57e459ff2bf/users/4217c49e92714b09a7f76ec096d97e31/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:44:18.783228 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1ce21-8732-4104-8ed5-69e6780b1fab admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:18.793522 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1ce21-8732-4104-8ed5-69e6780b1fab admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:18.848980 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eefada-19c8-4538-913c-614bec9d6fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:18.849606 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eefada-19c8-4538-913c-614bec9d6fab admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:18.849842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eefada-19c8-4538-913c-614bec9d6fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:18.850114 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eefada-19c8-4538-913c-614bec9d6fab admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:18.850738 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eefada-19c8-4538-913c-614bec9d6fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:18.852345 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eefada-19c8-4538-913c-614bec9d6fab admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:18.861835 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1268/2516] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:44:18.866221 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99de41-6942-445a-82ac-8b56507858ea admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:18.876673 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99de41-6942-445a-82ac-8b56507858ea admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:18.882703 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fb101-6c46-44ed-8086-2cc4554ebed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:18.883351 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fb101-6c46-44ed-8086-2cc4554ebed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:18.883625 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fb101-6c46-44ed-8086-2cc4554ebed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:18.883922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fb101-6c46-44ed-8086-2cc4554ebed2 admin admin] PATH_INFO: `/v3/projects/d2bbb067e3dc42c9ac05e57e459ff2bf/users/4217c49e92714b09a7f76ec096d97e31/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:18.885878 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fb101-6c46-44ed-8086-2cc4554ebed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2bbb067e3dc42c9ac05e57e459ff2bf, user_id=4217c49e92714b09a7f76ec096d97e31, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:18.888459 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fb101-6c46-44ed-8086-2cc4554ebed2 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:18.899300 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1249/2517] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:44:18 2022] PUT /identity/v3/projects/d2bbb067e3dc42c9ac05e57e459ff2bf/users/4217c49e92714b09a7f76ec096d97e31/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:44:18.910476 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a06a51-3e7f-461d-9fb9-07d8bbc83997 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:18.910765 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a06a51-3e7f-461d-9fb9-07d8bbc83997 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:18.911039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a06a51-3e7f-461d-9fb9-07d8bbc83997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:18.939025 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-21a06a51-3e7f-461d-9fb9-07d8bbc83997 None None] MFA Rules not processed for user `4217c49e92714b09a7f76ec096d97e31`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:44:18.940161 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-21a06a51-3e7f-461d-9fb9-07d8bbc83997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:18.980240 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:44:19.008622 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1269/2518] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:44:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:44:19.020266 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fb101-6c46-44ed-8086-2cc4554ebed2 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:19.021988 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-512fb101-6c46-44ed-8086-2cc4554ebed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:19.078927 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fb101-6c46-44ed-8086-2cc4554ebed2 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:19.083148 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b8846-0c53-4396-8cf8-2dc586d31796 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:19.083736 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b8846-0c53-4396-8cf8-2dc586d31796 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:19.084012 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b8846-0c53-4396-8cf8-2dc586d31796 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:19.084280 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b8846-0c53-4396-8cf8-2dc586d31796 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:19.093065 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b8846-0c53-4396-8cf8-2dc586d31796 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:19.094470 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b8846-0c53-4396-8cf8-2dc586d31796 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:19.104355 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1250/2519] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:44:19 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:44:19.513320 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a06a51-3e7f-461d-9fb9-07d8bbc83997 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:19.575282 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a06a51-3e7f-461d-9fb9-07d8bbc83997 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:19.579764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8211f47-dfc1-4f37-86e3-68a8c7a40ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:19.580386 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8211f47-dfc1-4f37-86e3-68a8c7a40ad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:19.580663 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8211f47-dfc1-4f37-86e3-68a8c7a40ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:19.580966 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8211f47-dfc1-4f37-86e3-68a8c7a40ad2 admin admin] PATH_INFO: `/v3/users/47815b526b0e4f42868fa7e6d03ec098` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:19.582723 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8211f47-dfc1-4f37-86e3-68a8c7a40ad2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47815b526b0e4f42868fa7e6d03ec098)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:19.584617 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8211f47-dfc1-4f37-86e3-68a8c7a40ad2 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:19.630564 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-e8211f47-dfc1-4f37-86e3-68a8c7a40ad2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47815b526b0e4f42868fa7e6d03ec098'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:19.631267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-e8211f47-dfc1-4f37-86e3-68a8c7a40ad2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47815b526b0e4f42868fa7e6d03ec098'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:19.637910 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-e8211f47-dfc1-4f37-86e3-68a8c7a40ad2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47815b526b0e4f42868fa7e6d03ec098'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:19.650687 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-e8211f47-dfc1-4f37-86e3-68a8c7a40ad2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47815b526b0e4f42868fa7e6d03ec098'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:19.651326 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-e8211f47-dfc1-4f37-86e3-68a8c7a40ad2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47815b526b0e4f42868fa7e6d03ec098'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:19.663520 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1270/2520] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:44:19 2022] DELETE /identity/v3/users/47815b526b0e4f42868fa7e6d03ec098 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:44:19.668016 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12e23f-c3dd-422b-8baa-c7733a5670c7 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:19.668502 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12e23f-c3dd-422b-8baa-c7733a5670c7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:19.668612 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12e23f-c3dd-422b-8baa-c7733a5670c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:19.688012 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-2c12e23f-c3dd-422b-8baa-c7733a5670c7 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:44:19.689232 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2c12e23f-c3dd-422b-8baa-c7733a5670c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:19.707629 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:44:19.748375 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1251/2521] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:44:19.758688 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8211f47-dfc1-4f37-86e3-68a8c7a40ad2 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:19.760129 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e8211f47-dfc1-4f37-86e3-68a8c7a40ad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:19.815401 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8211f47-dfc1-4f37-86e3-68a8c7a40ad2 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:19.819620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014593d-122a-4096-a74d-a03bf96c549f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:19.820221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014593d-122a-4096-a74d-a03bf96c549f service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:19.820500 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014593d-122a-4096-a74d-a03bf96c549f service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:19.820811 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014593d-122a-4096-a74d-a03bf96c549f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:19.830555 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014593d-122a-4096-a74d-a03bf96c549f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:19.832239 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014593d-122a-4096-a74d-a03bf96c549f service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:19.836396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:44:19.844022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1271/2522] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:44:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:44:20.108002 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12e23f-c3dd-422b-8baa-c7733a5670c7 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:20.109434 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2c12e23f-c3dd-422b-8baa-c7733a5670c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:20.123687 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12e23f-c3dd-422b-8baa-c7733a5670c7 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:20.129768 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1b19f-1fc3-4bb0-b8e9-9e079708059f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:20.130455 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1b19f-1fc3-4bb0-b8e9-9e079708059f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:20.130666 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1b19f-1fc3-4bb0-b8e9-9e079708059f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:20.130979 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1b19f-1fc3-4bb0-b8e9-9e079708059f admin admin] PATH_INFO: `/v3/projects/bf0c9a48152a4ba2b265df31ec724ac5` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:20.132102 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f1b19f-1fc3-4bb0-b8e9-9e079708059f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf0c9a48152a4ba2b265df31ec724ac5)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:20.134531 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f1b19f-1fc3-4bb0-b8e9-9e079708059f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:20.182858 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e2f1b19f-1fc3-4bb0-b8e9-9e079708059f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf0c9a48152a4ba2b265df31ec724ac5', 'request_id': 'req-e2f1b19f-1fc3-4bb0-b8e9-9e079708059f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:20.188485 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e2f1b19f-1fc3-4bb0-b8e9-9e079708059f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf0c9a48152a4ba2b265df31ec724ac5', 'request_id': 'req-e2f1b19f-1fc3-4bb0-b8e9-9e079708059f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:20.195030 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e2f1b19f-1fc3-4bb0-b8e9-9e079708059f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf0c9a48152a4ba2b265df31ec724ac5', 'request_id': 'req-e2f1b19f-1fc3-4bb0-b8e9-9e079708059f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:20.207565 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e2f1b19f-1fc3-4bb0-b8e9-9e079708059f admin admin] The token cache is being invalidate because project bf0c9a48152a4ba2b265df31ec724ac5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:44:20.207859 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e2f1b19f-1fc3-4bb0-b8e9-9e079708059f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:20.208484 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e2f1b19f-1fc3-4bb0-b8e9-9e079708059f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:20.210041 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1252/2523] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:44:20 2022] DELETE /identity/v3/projects/bf0c9a48152a4ba2b265df31ec724ac5 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:44:20.214096 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014593d-122a-4096-a74d-a03bf96c549f service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:20.224354 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014593d-122a-4096-a74d-a03bf96c549f service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:20.279548 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b071ab2-408e-46df-8b88-48292948d7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:20.280095 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b071ab2-408e-46df-8b88-48292948d7eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:20.280373 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b071ab2-408e-46df-8b88-48292948d7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:20.280645 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b071ab2-408e-46df-8b88-48292948d7eb admin admin] PATH_INFO: `/v3/users/4217c49e92714b09a7f76ec096d97e31` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:20.281808 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b071ab2-408e-46df-8b88-48292948d7eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4217c49e92714b09a7f76ec096d97e31)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:20.283707 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b071ab2-408e-46df-8b88-48292948d7eb admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:20.330679 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6b071ab2-408e-46df-8b88-48292948d7eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4217c49e92714b09a7f76ec096d97e31'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:20.331312 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6b071ab2-408e-46df-8b88-48292948d7eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4217c49e92714b09a7f76ec096d97e31'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:20.338366 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6b071ab2-408e-46df-8b88-48292948d7eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4217c49e92714b09a7f76ec096d97e31'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:20.351491 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6b071ab2-408e-46df-8b88-48292948d7eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4217c49e92714b09a7f76ec096d97e31'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:20.352121 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6b071ab2-408e-46df-8b88-48292948d7eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4217c49e92714b09a7f76ec096d97e31'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:20.364540 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1272/2524] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:44:20 2022] DELETE /identity/v3/users/4217c49e92714b09a7f76ec096d97e31 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:44:20.632103 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1b19f-1fc3-4bb0-b8e9-9e079708059f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:20.633629 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e2f1b19f-1fc3-4bb0-b8e9-9e079708059f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:20.648675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1b19f-1fc3-4bb0-b8e9-9e079708059f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:20.705853 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f7983-492a-48de-8b65-c94cd32f8b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:20.706481 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f7983-492a-48de-8b65-c94cd32f8b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:20.706783 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f7983-492a-48de-8b65-c94cd32f8b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:20.707056 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f7983-492a-48de-8b65-c94cd32f8b70 admin admin] PATH_INFO: `/v3/projects/d2bbb067e3dc42c9ac05e57e459ff2bf` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:20.708174 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f7983-492a-48de-8b65-c94cd32f8b70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2bbb067e3dc42c9ac05e57e459ff2bf)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:20.710082 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f7983-492a-48de-8b65-c94cd32f8b70 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:20.757748 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-611f7983-492a-48de-8b65-c94cd32f8b70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2bbb067e3dc42c9ac05e57e459ff2bf', 'request_id': 'req-611f7983-492a-48de-8b65-c94cd32f8b70'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:20.758906 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:44:20.764621 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-611f7983-492a-48de-8b65-c94cd32f8b70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2bbb067e3dc42c9ac05e57e459ff2bf', 'request_id': 'req-611f7983-492a-48de-8b65-c94cd32f8b70'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:20.772361 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-611f7983-492a-48de-8b65-c94cd32f8b70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2bbb067e3dc42c9ac05e57e459ff2bf', 'request_id': 'req-611f7983-492a-48de-8b65-c94cd32f8b70'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:20.784351 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-611f7983-492a-48de-8b65-c94cd32f8b70 admin admin] The token cache is being invalidate because project d2bbb067e3dc42c9ac05e57e459ff2bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:44:20.785163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-611f7983-492a-48de-8b65-c94cd32f8b70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:20.786097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-611f7983-492a-48de-8b65-c94cd32f8b70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:20.787584 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1253/2525] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:44:20 2022] DELETE /identity/v3/projects/d2bbb067e3dc42c9ac05e57e459ff2bf => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:44:20.792359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceba4b0-064c-44ef-a833-2a060c2ac602 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:20.792646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceba4b0-064c-44ef-a833-2a060c2ac602 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:20.792964 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceba4b0-064c-44ef-a833-2a060c2ac602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:20.812644 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-0ceba4b0-064c-44ef-a833-2a060c2ac602 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:44:20.813792 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-0ceba4b0-064c-44ef-a833-2a060c2ac602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:20.870799 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1273/2526] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:44:20.878391 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091536b-7400-4158-8320-f4f045d28c3d None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:20.878671 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091536b-7400-4158-8320-f4f045d28c3d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:20.878941 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091536b-7400-4158-8320-f4f045d28c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:20.898630 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c091536b-7400-4158-8320-f4f045d28c3d None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:44:20.899830 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c091536b-7400-4158-8320-f4f045d28c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:20.907572 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1254/2527] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:44:20.912482 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceba4b0-064c-44ef-a833-2a060c2ac602 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:20.922926 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceba4b0-064c-44ef-a833-2a060c2ac602 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:20.928312 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf61f8e-e698-4599-8a5b-8d1649eaf241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:20.928941 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf61f8e-e698-4599-8a5b-8d1649eaf241 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:20.929219 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf61f8e-e698-4599-8a5b-8d1649eaf241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:20.929488 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf61f8e-e698-4599-8a5b-8d1649eaf241 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:20.930123 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf61f8e-e698-4599-8a5b-8d1649eaf241 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:44:20.930446 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf61f8e-e698-4599-8a5b-8d1649eaf241 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:20.932069 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf61f8e-e698-4599-8a5b-8d1649eaf241 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:20.943718 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1274/2528] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:44:20.948119 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091536b-7400-4158-8320-f4f045d28c3d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:20.957409 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091536b-7400-4158-8320-f4f045d28c3d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:20.962098 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fc167-98c4-471e-ad20-da50479c80ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:20.962689 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fc167-98c4-471e-ad20-da50479c80ff admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:20.962973 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fc167-98c4-471e-ad20-da50479c80ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:20.963248 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fc167-98c4-471e-ad20-da50479c80ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:20.963874 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510fc167-98c4-471e-ad20-da50479c80ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:20.965804 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510fc167-98c4-471e-ad20-da50479c80ff admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:20.994913 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1255/2529] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:44:20 2022] POST /identity/v3/projects => generated 357 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:44:20.998898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf61f8e-e698-4599-8a5b-8d1649eaf241 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:21.005869 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:44:21.009834 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf61f8e-e698-4599-8a5b-8d1649eaf241 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:21.070120 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b70732-2fd9-4a0d-9e96-dc689a51ec2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:21.070647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b70732-2fd9-4a0d-9e96-dc689a51ec2a admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:21.070926 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b70732-2fd9-4a0d-9e96-dc689a51ec2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:21.071212 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b70732-2fd9-4a0d-9e96-dc689a51ec2a admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:21.071831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b70732-2fd9-4a0d-9e96-dc689a51ec2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:21.073772 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b70732-2fd9-4a0d-9e96-dc689a51ec2a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:21.096457 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1275/2530] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:44:20 2022] POST /identity/v3/users => generated 279 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:44:21.100768 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fc167-98c4-471e-ad20-da50479c80ff admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:21.111124 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fc167-98c4-471e-ad20-da50479c80ff admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:21.115771 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea09e3-398a-44d1-8772-2d0ea2c12053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:21.116358 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea09e3-398a-44d1-8772-2d0ea2c12053 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:21.116631 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea09e3-398a-44d1-8772-2d0ea2c12053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:21.117038 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea09e3-398a-44d1-8772-2d0ea2c12053 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:21.117670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea09e3-398a-44d1-8772-2d0ea2c12053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:21.119277 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea09e3-398a-44d1-8772-2d0ea2c12053 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:21.129773 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1256/2531] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:44:21.133592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b70732-2fd9-4a0d-9e96-dc689a51ec2a admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:21.143230 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b70732-2fd9-4a0d-9e96-dc689a51ec2a admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:21.148322 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009e82c-e3f1-44b9-99aa-29241050be4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:21.148935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009e82c-e3f1-44b9-99aa-29241050be4e admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:21.149212 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009e82c-e3f1-44b9-99aa-29241050be4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:21.149478 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009e82c-e3f1-44b9-99aa-29241050be4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:21.150096 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e009e82c-e3f1-44b9-99aa-29241050be4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:21.151654 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e009e82c-e3f1-44b9-99aa-29241050be4e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:21.161231 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1276/2532] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:44:21.165201 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea09e3-398a-44d1-8772-2d0ea2c12053 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:21.175012 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea09e3-398a-44d1-8772-2d0ea2c12053 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:21.180084 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675b20b-410b-43b1-9a38-ff601d219c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:21.180751 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675b20b-410b-43b1-9a38-ff601d219c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:21.181032 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675b20b-410b-43b1-9a38-ff601d219c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:21.181302 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675b20b-410b-43b1-9a38-ff601d219c0f admin admin] PATH_INFO: `/v3/projects/8ef82a1de0124ad2b692d16c9015dc0c/users/81df023f187e4947b5f66ffef8de0eaf/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:21.195513 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675b20b-410b-43b1-9a38-ff601d219c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ef82a1de0124ad2b692d16c9015dc0c, user_id=81df023f187e4947b5f66ffef8de0eaf, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:21.198174 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675b20b-410b-43b1-9a38-ff601d219c0f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:21.211121 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1257/2533] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:44:21 2022] PUT /identity/v3/projects/8ef82a1de0124ad2b692d16c9015dc0c/users/81df023f187e4947b5f66ffef8de0eaf/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:44:21.222106 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11380910-a47b-4dea-8066-437f57b488ff None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:21.222630 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11380910-a47b-4dea-8066-437f57b488ff None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:21.222905 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11380910-a47b-4dea-8066-437f57b488ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:21.251317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-11380910-a47b-4dea-8066-437f57b488ff None None] MFA Rules not processed for user `81df023f187e4947b5f66ffef8de0eaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:44:21.252507 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-11380910-a47b-4dea-8066-437f57b488ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:21.315132 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1277/2534] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:44:21 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:44:21.320216 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675b20b-410b-43b1-9a38-ff601d219c0f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:21.380776 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675b20b-410b-43b1-9a38-ff601d219c0f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:21.385325 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662bb063-7407-42ff-a0f1-5f7036d1e3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:21.385955 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662bb063-7407-42ff-a0f1-5f7036d1e3cd admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:21.386236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662bb063-7407-42ff-a0f1-5f7036d1e3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:21.386508 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662bb063-7407-42ff-a0f1-5f7036d1e3cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:21.387126 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662bb063-7407-42ff-a0f1-5f7036d1e3cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:21.389057 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662bb063-7407-42ff-a0f1-5f7036d1e3cd admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:21.414488 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1258/2535] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:44:21 2022] POST /identity/v3/projects => generated 351 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:44:21.418548 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11380910-a47b-4dea-8066-437f57b488ff None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:21.429552 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11380910-a47b-4dea-8066-437f57b488ff None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:21.483395 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3fb8d-1c6a-4384-a49a-01378dfc19e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:21.483992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3fb8d-1c6a-4384-a49a-01378dfc19e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:21.484267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3fb8d-1c6a-4384-a49a-01378dfc19e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:21.484540 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3fb8d-1c6a-4384-a49a-01378dfc19e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:21.485184 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba3fb8d-1c6a-4384-a49a-01378dfc19e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:21.487072 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba3fb8d-1c6a-4384-a49a-01378dfc19e0 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:21.509080 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1278/2536] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:44:21 2022] POST /identity/v3/users => generated 282 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:44:21.513232 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662bb063-7407-42ff-a0f1-5f7036d1e3cd admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:21.523498 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662bb063-7407-42ff-a0f1-5f7036d1e3cd admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:21.528276 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ecd17-450c-44c8-a0f5-c1a1feac9cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:21.528899 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ecd17-450c-44c8-a0f5-c1a1feac9cbe admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:21.529175 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ecd17-450c-44c8-a0f5-c1a1feac9cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:21.529441 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ecd17-450c-44c8-a0f5-c1a1feac9cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:21.530073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ecd17-450c-44c8-a0f5-c1a1feac9cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:21.531713 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ecd17-450c-44c8-a0f5-c1a1feac9cbe admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:21.542732 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1259/2537] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:44:21.546906 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3fb8d-1c6a-4384-a49a-01378dfc19e0 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:21.552675 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:44:21.558770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3fb8d-1c6a-4384-a49a-01378dfc19e0 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:21.563459 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00b0c7-46ab-468c-aa5c-3f9670b7f6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:21.564099 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00b0c7-46ab-468c-aa5c-3f9670b7f6d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:21.564374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00b0c7-46ab-468c-aa5c-3f9670b7f6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:21.564666 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00b0c7-46ab-468c-aa5c-3f9670b7f6d8 admin admin] PATH_INFO: `/v3/projects/c7a5c03331cd439aada4994ae41f9956/users/9800c8f0b71c42d09b4e2296bcabc0cf/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:21.579169 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff00b0c7-46ab-468c-aa5c-3f9670b7f6d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7a5c03331cd439aada4994ae41f9956, user_id=9800c8f0b71c42d09b4e2296bcabc0cf, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:21.581807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff00b0c7-46ab-468c-aa5c-3f9670b7f6d8 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:21.593753 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1279/2538] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:44:21 2022] PUT /identity/v3/projects/c7a5c03331cd439aada4994ae41f9956/users/9800c8f0b71c42d09b4e2296bcabc0cf/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:44:21.598209 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ecd17-450c-44c8-a0f5-c1a1feac9cbe admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:21.608755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ecd17-450c-44c8-a0f5-c1a1feac9cbe admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:21.654000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:44:21.667635 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef73aa5-3940-472a-b02c-7d8c2f444dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:21.668234 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef73aa5-3940-472a-b02c-7d8c2f444dbd admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:21.668514 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef73aa5-3940-472a-b02c-7d8c2f444dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:21.668833 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef73aa5-3940-472a-b02c-7d8c2f444dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:21.669486 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef73aa5-3940-472a-b02c-7d8c2f444dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:21.671059 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef73aa5-3940-472a-b02c-7d8c2f444dbd admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:21.681073 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1260/2539] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:44:21.685310 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00b0c7-46ab-468c-aa5c-3f9670b7f6d8 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:21.695516 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00b0c7-46ab-468c-aa5c-3f9670b7f6d8 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:21.700052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d312136-bc55-483a-b553-027458fe4ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:21.700686 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d312136-bc55-483a-b553-027458fe4ef5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:21.700984 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d312136-bc55-483a-b553-027458fe4ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:21.701251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d312136-bc55-483a-b553-027458fe4ef5 admin admin] PATH_INFO: `/v3/projects/c7a5c03331cd439aada4994ae41f9956/users/9800c8f0b71c42d09b4e2296bcabc0cf/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:21.703122 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d312136-bc55-483a-b553-027458fe4ef5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7a5c03331cd439aada4994ae41f9956, user_id=9800c8f0b71c42d09b4e2296bcabc0cf, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:21.705728 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d312136-bc55-483a-b553-027458fe4ef5 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:21.716377 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1280/2540] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:44:21 2022] PUT /identity/v3/projects/c7a5c03331cd439aada4994ae41f9956/users/9800c8f0b71c42d09b4e2296bcabc0cf/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:44:21.726907 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9e82c-72d7-450a-8058-b3a0f482ca98 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:21.727378 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9e82c-72d7-450a-8058-b3a0f482ca98 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:21.727474 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9e82c-72d7-450a-8058-b3a0f482ca98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:21.755723 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-d3c9e82c-72d7-450a-8058-b3a0f482ca98 None None] MFA Rules not processed for user `9800c8f0b71c42d09b4e2296bcabc0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:44:21.756948 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d3c9e82c-72d7-450a-8058-b3a0f482ca98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:21.822541 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1261/2541] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:44:21 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:44:21.833883 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d312136-bc55-483a-b553-027458fe4ef5 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:21.835524 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9d312136-bc55-483a-b553-027458fe4ef5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:21.892578 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d312136-bc55-483a-b553-027458fe4ef5 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:21.897702 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f43fa9-beca-4106-a448-83172cd45aaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:21.898303 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f43fa9-beca-4106-a448-83172cd45aaa service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:21.898781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f43fa9-beca-4106-a448-83172cd45aaa service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:21.899101 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f43fa9-beca-4106-a448-83172cd45aaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:21.909215 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f43fa9-beca-4106-a448-83172cd45aaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:21.911246 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f43fa9-beca-4106-a448-83172cd45aaa service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:21.920396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1281/2542] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:44:21 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:44:21.994009 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9e82c-72d7-450a-8058-b3a0f482ca98 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:22.003924 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9e82c-72d7-450a-8058-b3a0f482ca98 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:22.007490 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccca06f-9c4e-4aab-9d92-afe799621048 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:22.008085 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccca06f-9c4e-4aab-9d92-afe799621048 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:22.008361 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccca06f-9c4e-4aab-9d92-afe799621048 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:22.008627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccca06f-9c4e-4aab-9d92-afe799621048 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:22.064834 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccca06f-9c4e-4aab-9d92-afe799621048 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:22.066275 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccca06f-9c4e-4aab-9d92-afe799621048 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:22.075157 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1262/2543] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:44:21 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:44:31.916257 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=40,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:44:32.071202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=69 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:44:36.871339 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f43fa9-beca-4106-a448-83172cd45aaa service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:36.932796 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f43fa9-beca-4106-a448-83172cd45aaa service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:36.937237 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4d921-5676-464a-869f-1bbdb5b9d896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:36.937848 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4d921-5676-464a-869f-1bbdb5b9d896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:36.938121 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4d921-5676-464a-869f-1bbdb5b9d896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:36.938386 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4d921-5676-464a-869f-1bbdb5b9d896 admin admin] PATH_INFO: `/v3/users/81df023f187e4947b5f66ffef8de0eaf` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:36.939486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c4d921-5676-464a-869f-1bbdb5b9d896 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81df023f187e4947b5f66ffef8de0eaf)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:36.941417 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c4d921-5676-464a-869f-1bbdb5b9d896 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:36.984822 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b7c4d921-5676-464a-869f-1bbdb5b9d896 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81df023f187e4947b5f66ffef8de0eaf'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:36.985463 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b7c4d921-5676-464a-869f-1bbdb5b9d896 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81df023f187e4947b5f66ffef8de0eaf'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:36.993233 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b7c4d921-5676-464a-869f-1bbdb5b9d896 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81df023f187e4947b5f66ffef8de0eaf'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:37.006396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b7c4d921-5676-464a-869f-1bbdb5b9d896 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81df023f187e4947b5f66ffef8de0eaf'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:37.007022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b7c4d921-5676-464a-869f-1bbdb5b9d896 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81df023f187e4947b5f66ffef8de0eaf'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:37.019003 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1282/2544] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:44:36 2022] DELETE /identity/v3/users/81df023f187e4947b5f66ffef8de0eaf => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:44:37.023580 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b1560-edea-4180-a880-dd1352a11bd6 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:37.024342 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b1560-edea-4180-a880-dd1352a11bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:37.024619 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b1560-edea-4180-a880-dd1352a11bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:37.044095 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c79b1560-edea-4180-a880-dd1352a11bd6 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:44:37.045323 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c79b1560-edea-4180-a880-dd1352a11bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:37.104051 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1263/2545] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:44:37.113643 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4d921-5676-464a-869f-1bbdb5b9d896 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:37.115606 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b7c4d921-5676-464a-869f-1bbdb5b9d896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:37.172235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4d921-5676-464a-869f-1bbdb5b9d896 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:37.175842 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e2220-e952-438e-8db6-abc4c6cce2cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:37.176442 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e2220-e952-438e-8db6-abc4c6cce2cf service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:37.177280 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e2220-e952-438e-8db6-abc4c6cce2cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:37.177550 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e2220-e952-438e-8db6-abc4c6cce2cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:37.186344 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e2220-e952-438e-8db6-abc4c6cce2cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:37.187766 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e2220-e952-438e-8db6-abc4c6cce2cf service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:37.197632 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1283/2546] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:44:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:44:37.464332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b1560-edea-4180-a880-dd1352a11bd6 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:37.465857 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c79b1560-edea-4180-a880-dd1352a11bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:37.478291 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b1560-edea-4180-a880-dd1352a11bd6 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:37.483402 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60651e9b-0908-45a3-9a93-85a7319f0690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:37.484042 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60651e9b-0908-45a3-9a93-85a7319f0690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:37.484321 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60651e9b-0908-45a3-9a93-85a7319f0690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:37.484596 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60651e9b-0908-45a3-9a93-85a7319f0690 admin admin] PATH_INFO: `/v3/projects/8ef82a1de0124ad2b692d16c9015dc0c` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:37.485751 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60651e9b-0908-45a3-9a93-85a7319f0690 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ef82a1de0124ad2b692d16c9015dc0c)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:37.487670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60651e9b-0908-45a3-9a93-85a7319f0690 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:37.534438 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-60651e9b-0908-45a3-9a93-85a7319f0690 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ef82a1de0124ad2b692d16c9015dc0c', 'request_id': 'req-60651e9b-0908-45a3-9a93-85a7319f0690'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:37.540575 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-60651e9b-0908-45a3-9a93-85a7319f0690 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ef82a1de0124ad2b692d16c9015dc0c', 'request_id': 'req-60651e9b-0908-45a3-9a93-85a7319f0690'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:37.546808 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-60651e9b-0908-45a3-9a93-85a7319f0690 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ef82a1de0124ad2b692d16c9015dc0c', 'request_id': 'req-60651e9b-0908-45a3-9a93-85a7319f0690'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:37.559614 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-60651e9b-0908-45a3-9a93-85a7319f0690 admin admin] The token cache is being invalidate because project 8ef82a1de0124ad2b692d16c9015dc0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:44:37.559922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-60651e9b-0908-45a3-9a93-85a7319f0690 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:37.560541 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-60651e9b-0908-45a3-9a93-85a7319f0690 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:37.562030 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1264/2547] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:44:37 2022] DELETE /identity/v3/projects/8ef82a1de0124ad2b692d16c9015dc0c => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:44:37.565965 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e2220-e952-438e-8db6-abc4c6cce2cf service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:37.576277 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e2220-e952-438e-8db6-abc4c6cce2cf service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:37.594515 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:44:37.634742 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b0094-19e2-4def-b585-cfd48960b3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:37.635480 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b0094-19e2-4def-b585-cfd48960b3a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:37.635763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b0094-19e2-4def-b585-cfd48960b3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:37.636035 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b0094-19e2-4def-b585-cfd48960b3a6 admin admin] PATH_INFO: `/v3/users/9800c8f0b71c42d09b4e2296bcabc0cf` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:37.637194 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b0094-19e2-4def-b585-cfd48960b3a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9800c8f0b71c42d09b4e2296bcabc0cf)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:37.639121 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b0094-19e2-4def-b585-cfd48960b3a6 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:37.684525 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f72b0094-19e2-4def-b585-cfd48960b3a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9800c8f0b71c42d09b4e2296bcabc0cf'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:37.685208 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f72b0094-19e2-4def-b585-cfd48960b3a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9800c8f0b71c42d09b4e2296bcabc0cf'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:37.692143 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f72b0094-19e2-4def-b585-cfd48960b3a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9800c8f0b71c42d09b4e2296bcabc0cf'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:37.705108 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f72b0094-19e2-4def-b585-cfd48960b3a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9800c8f0b71c42d09b4e2296bcabc0cf'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:37.705742 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f72b0094-19e2-4def-b585-cfd48960b3a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9800c8f0b71c42d09b4e2296bcabc0cf'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:37.718000 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1284/2548] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:44:37 2022] DELETE /identity/v3/users/9800c8f0b71c42d09b4e2296bcabc0cf => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:44:37.985512 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60651e9b-0908-45a3-9a93-85a7319f0690 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:37.987007 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-60651e9b-0908-45a3-9a93-85a7319f0690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:38.002648 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60651e9b-0908-45a3-9a93-85a7319f0690 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:38.062207 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2882d25-9535-427d-810e-9ac3e0bfd8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:38.062851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2882d25-9535-427d-810e-9ac3e0bfd8a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:38.063093 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2882d25-9535-427d-810e-9ac3e0bfd8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:38.063388 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2882d25-9535-427d-810e-9ac3e0bfd8a5 admin admin] PATH_INFO: `/v3/projects/c7a5c03331cd439aada4994ae41f9956` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:38.064480 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2882d25-9535-427d-810e-9ac3e0bfd8a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7a5c03331cd439aada4994ae41f9956)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:38.066400 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2882d25-9535-427d-810e-9ac3e0bfd8a5 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:38.099785 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:44:38.117814 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b2882d25-9535-427d-810e-9ac3e0bfd8a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7a5c03331cd439aada4994ae41f9956', 'request_id': 'req-b2882d25-9535-427d-810e-9ac3e0bfd8a5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:38.123187 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b2882d25-9535-427d-810e-9ac3e0bfd8a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7a5c03331cd439aada4994ae41f9956', 'request_id': 'req-b2882d25-9535-427d-810e-9ac3e0bfd8a5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:38.129566 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b2882d25-9535-427d-810e-9ac3e0bfd8a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7a5c03331cd439aada4994ae41f9956', 'request_id': 'req-b2882d25-9535-427d-810e-9ac3e0bfd8a5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:38.141814 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b2882d25-9535-427d-810e-9ac3e0bfd8a5 admin admin] The token cache is being invalidate because project c7a5c03331cd439aada4994ae41f9956 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:44:38.142102 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b2882d25-9535-427d-810e-9ac3e0bfd8a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:38.142713 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b2882d25-9535-427d-810e-9ac3e0bfd8a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:44:38.144176 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1265/2549] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:44:37 2022] DELETE /identity/v3/projects/c7a5c03331cd439aada4994ae41f9956 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:44:38.149301 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e172f-e4b1-4c29-8ac1-73f3b91b6748 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:38.150115 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e172f-e4b1-4c29-8ac1-73f3b91b6748 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:38.150385 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e172f-e4b1-4c29-8ac1-73f3b91b6748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:38.172324 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-958e172f-e4b1-4c29-8ac1-73f3b91b6748 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:44:38.173743 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-958e172f-e4b1-4c29-8ac1-73f3b91b6748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:38.231906 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1285/2550] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:44:38.241499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025aa69-df0f-4ce0-9b43-d5d288a78068 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:38.241781 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025aa69-df0f-4ce0-9b43-d5d288a78068 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:38.242053 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025aa69-df0f-4ce0-9b43-d5d288a78068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:38.261578 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-b025aa69-df0f-4ce0-9b43-d5d288a78068 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:44:38.262774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b025aa69-df0f-4ce0-9b43-d5d288a78068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:38.270629 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1266/2551] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:44:38.274751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e172f-e4b1-4c29-8ac1-73f3b91b6748 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:38.284309 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e172f-e4b1-4c29-8ac1-73f3b91b6748 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:38.289255 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28c1ae-a3b9-4b92-bea8-067a02396b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:38.289872 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28c1ae-a3b9-4b92-bea8-067a02396b85 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:38.290145 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28c1ae-a3b9-4b92-bea8-067a02396b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:38.290408 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28c1ae-a3b9-4b92-bea8-067a02396b85 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:38.291033 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28c1ae-a3b9-4b92-bea8-067a02396b85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:44:38.291329 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28c1ae-a3b9-4b92-bea8-067a02396b85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:38.292935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28c1ae-a3b9-4b92-bea8-067a02396b85 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:38.308479 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1286/2552] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:44:38.309744 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025aa69-df0f-4ce0-9b43-d5d288a78068 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:38.318940 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025aa69-df0f-4ce0-9b43-d5d288a78068 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:38.323563 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595fd9c-dadf-462e-8db7-9b5475a5a9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:38.324152 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595fd9c-dadf-462e-8db7-9b5475a5a9c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:38.324429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595fd9c-dadf-462e-8db7-9b5475a5a9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:38.324726 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595fd9c-dadf-462e-8db7-9b5475a5a9c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:38.325349 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595fd9c-dadf-462e-8db7-9b5475a5a9c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:38.327233 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595fd9c-dadf-462e-8db7-9b5475a5a9c6 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:38.354900 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1267/2553] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:44:38 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:44:38.358917 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28c1ae-a3b9-4b92-bea8-067a02396b85 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:38.369584 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28c1ae-a3b9-4b92-bea8-067a02396b85 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:38.397057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:44:38.429476 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af21448-34b5-4cdf-9992-678a8b32e980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:38.430014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af21448-34b5-4cdf-9992-678a8b32e980 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:38.430290 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af21448-34b5-4cdf-9992-678a8b32e980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:38.430561 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af21448-34b5-4cdf-9992-678a8b32e980 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:38.431191 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af21448-34b5-4cdf-9992-678a8b32e980 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:38.433134 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af21448-34b5-4cdf-9992-678a8b32e980 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:38.454614 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1287/2554] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:44:38 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:44:38.459676 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595fd9c-dadf-462e-8db7-9b5475a5a9c6 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:38.469911 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595fd9c-dadf-462e-8db7-9b5475a5a9c6 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:38.475648 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e7183-c8c2-4e56-9606-1e294fbb5765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:38.476328 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e7183-c8c2-4e56-9606-1e294fbb5765 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:38.476604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e7183-c8c2-4e56-9606-1e294fbb5765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:38.476903 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e7183-c8c2-4e56-9606-1e294fbb5765 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:38.477541 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e7183-c8c2-4e56-9606-1e294fbb5765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:38.479117 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e7183-c8c2-4e56-9606-1e294fbb5765 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:38.488912 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1268/2555] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:44:38.492974 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af21448-34b5-4cdf-9992-678a8b32e980 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:38.502947 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af21448-34b5-4cdf-9992-678a8b32e980 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:38.507530 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab7a46-ba2e-4b4f-804f-54c0174c0f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:38.508111 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab7a46-ba2e-4b4f-804f-54c0174c0f21 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:38.508383 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab7a46-ba2e-4b4f-804f-54c0174c0f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:38.508649 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab7a46-ba2e-4b4f-804f-54c0174c0f21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:38.509304 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab7a46-ba2e-4b4f-804f-54c0174c0f21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:38.510855 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab7a46-ba2e-4b4f-804f-54c0174c0f21 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:38.524240 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1288/2556] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:44:38.525443 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e7183-c8c2-4e56-9606-1e294fbb5765 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:38.535536 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e7183-c8c2-4e56-9606-1e294fbb5765 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:38.540899 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bc56a-7e6d-4fdb-8ac5-3a003027051b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:38.541542 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bc56a-7e6d-4fdb-8ac5-3a003027051b admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:38.541819 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bc56a-7e6d-4fdb-8ac5-3a003027051b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:38.542090 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bc56a-7e6d-4fdb-8ac5-3a003027051b admin admin] PATH_INFO: `/v3/projects/ae121132e9634b93accbef6500c11681/users/b22b4c04eeea48baa623160930fdb3d6/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:38.556445 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3bc56a-7e6d-4fdb-8ac5-3a003027051b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae121132e9634b93accbef6500c11681, user_id=b22b4c04eeea48baa623160930fdb3d6, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:38.559090 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3bc56a-7e6d-4fdb-8ac5-3a003027051b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:38.571730 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1269/2557] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:44:38 2022] PUT /identity/v3/projects/ae121132e9634b93accbef6500c11681/users/b22b4c04eeea48baa623160930fdb3d6/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:44:38.581917 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c22ae9-70e5-4e01-ac7c-66abdc8d9f63 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:38.582230 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c22ae9-70e5-4e01-ac7c-66abdc8d9f63 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:38.582553 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c22ae9-70e5-4e01-ac7c-66abdc8d9f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:38.610314 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-22c22ae9-70e5-4e01-ac7c-66abdc8d9f63 None None] MFA Rules not processed for user `b22b4c04eeea48baa623160930fdb3d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:44:38.611516 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-22c22ae9-70e5-4e01-ac7c-66abdc8d9f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:38.679802 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1289/2558] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:44:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:44:38.684151 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bc56a-7e6d-4fdb-8ac5-3a003027051b admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:38.747801 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bc56a-7e6d-4fdb-8ac5-3a003027051b admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:38.752793 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f1613-7d6b-415f-a630-8cfce4a426a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:38.753392 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f1613-7d6b-415f-a630-8cfce4a426a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:38.753673 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f1613-7d6b-415f-a630-8cfce4a426a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:38.753946 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f1613-7d6b-415f-a630-8cfce4a426a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:38.754564 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267f1613-7d6b-415f-a630-8cfce4a426a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:38.756781 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267f1613-7d6b-415f-a630-8cfce4a426a6 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:38.782743 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1270/2559] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:44:38 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:44:38.787274 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c22ae9-70e5-4e01-ac7c-66abdc8d9f63 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:38.797741 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c22ae9-70e5-4e01-ac7c-66abdc8d9f63 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:38.853469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e0584-0374-488e-9a06-cfb9de3a2bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:38.854075 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e0584-0374-488e-9a06-cfb9de3a2bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:38.854452 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e0584-0374-488e-9a06-cfb9de3a2bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:38.854724 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e0584-0374-488e-9a06-cfb9de3a2bc4 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:38.855377 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e0584-0374-488e-9a06-cfb9de3a2bc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:38.857353 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e0584-0374-488e-9a06-cfb9de3a2bc4 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:38.879336 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1290/2560] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:44:38 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:44:38.883560 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f1613-7d6b-415f-a630-8cfce4a426a6 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:38.893721 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f1613-7d6b-415f-a630-8cfce4a426a6 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:38.898547 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79cbb5-2161-4393-b16a-aadb2f3e5cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:38.899136 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79cbb5-2161-4393-b16a-aadb2f3e5cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:38.899415 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79cbb5-2161-4393-b16a-aadb2f3e5cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:38.899689 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79cbb5-2161-4393-b16a-aadb2f3e5cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:38.900346 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf79cbb5-2161-4393-b16a-aadb2f3e5cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:38.901973 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf79cbb5-2161-4393-b16a-aadb2f3e5cf5 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:38.913156 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1271/2561] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:44:38.917317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e0584-0374-488e-9a06-cfb9de3a2bc4 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:38.927933 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e0584-0374-488e-9a06-cfb9de3a2bc4 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:38.932573 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e85b0-3f0d-47aa-a8d0-ed9e641ea795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:38.933199 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e85b0-3f0d-47aa-a8d0-ed9e641ea795 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:38.933482 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e85b0-3f0d-47aa-a8d0-ed9e641ea795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:38.933761 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e85b0-3f0d-47aa-a8d0-ed9e641ea795 admin admin] PATH_INFO: `/v3/projects/80afbba338c34792a86e670f7c777ccd/users/3405adc2883c4267864d189e0157d758/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:38.948360 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e85b0-3f0d-47aa-a8d0-ed9e641ea795 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80afbba338c34792a86e670f7c777ccd, user_id=3405adc2883c4267864d189e0157d758, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:38.951056 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e85b0-3f0d-47aa-a8d0-ed9e641ea795 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:38.963866 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1291/2562] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:44:38 2022] PUT /identity/v3/projects/80afbba338c34792a86e670f7c777ccd/users/3405adc2883c4267864d189e0157d758/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:44:38.967860 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79cbb5-2161-4393-b16a-aadb2f3e5cf5 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:38.978403 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79cbb5-2161-4393-b16a-aadb2f3e5cf5 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:38.998358 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:44:39.035047 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526da36-070b-4f77-855c-7df751735325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:39.035645 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526da36-070b-4f77-855c-7df751735325 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:39.036406 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526da36-070b-4f77-855c-7df751735325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:39.036675 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526da36-070b-4f77-855c-7df751735325 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:39.037402 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526da36-070b-4f77-855c-7df751735325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:39.038980 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526da36-070b-4f77-855c-7df751735325 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:39.048365 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1272/2563] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:44:39.053781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e85b0-3f0d-47aa-a8d0-ed9e641ea795 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:39.059093 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:44:39.065077 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e85b0-3f0d-47aa-a8d0-ed9e641ea795 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:39.070183 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab96dff-bce9-4337-8cde-e4959330040b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:39.070832 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab96dff-bce9-4337-8cde-e4959330040b admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:39.071113 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab96dff-bce9-4337-8cde-e4959330040b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:39.071388 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab96dff-bce9-4337-8cde-e4959330040b admin admin] PATH_INFO: `/v3/projects/80afbba338c34792a86e670f7c777ccd/users/3405adc2883c4267864d189e0157d758/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:39.073425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab96dff-bce9-4337-8cde-e4959330040b admin admin] RBAC: Authorizing `identity:create_grant(project_id=80afbba338c34792a86e670f7c777ccd, user_id=3405adc2883c4267864d189e0157d758, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:39.076074 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab96dff-bce9-4337-8cde-e4959330040b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:39.086949 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1292/2564] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:44:39 2022] PUT /identity/v3/projects/80afbba338c34792a86e670f7c777ccd/users/3405adc2883c4267864d189e0157d758/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:44:39.096945 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb51982-1768-4c8b-953c-4856be2c22a8 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:39.097429 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb51982-1768-4c8b-953c-4856be2c22a8 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:39.097533 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb51982-1768-4c8b-953c-4856be2c22a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:39.125049 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-8bb51982-1768-4c8b-953c-4856be2c22a8 None None] MFA Rules not processed for user `3405adc2883c4267864d189e0157d758`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:44:39.126504 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8bb51982-1768-4c8b-953c-4856be2c22a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:39.192624 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1273/2565] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:44:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:44:39.203184 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab96dff-bce9-4337-8cde-e4959330040b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:39.204968 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5ab96dff-bce9-4337-8cde-e4959330040b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:39.263399 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab96dff-bce9-4337-8cde-e4959330040b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:39.266955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1265d71-d651-4449-8058-b7e4c93059cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:39.267545 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1265d71-d651-4449-8058-b7e4c93059cb service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:39.267817 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1265d71-d651-4449-8058-b7e4c93059cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:39.268082 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1265d71-d651-4449-8058-b7e4c93059cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:39.325332 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1265d71-d651-4449-8058-b7e4c93059cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:39.326766 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1265d71-d651-4449-8058-b7e4c93059cb service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:39.335545 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1293/2566] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:44:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:44:44.977702 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb51982-1768-4c8b-953c-4856be2c22a8 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:44:44.979094 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8bb51982-1768-4c8b-953c-4856be2c22a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:44:44.991089 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb51982-1768-4c8b-953c-4856be2c22a8 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:44:44.994576 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1c93f-6687-4b4e-acd1-70af325e90d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:44:44.995165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1c93f-6687-4b4e-acd1-70af325e90d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:44:44.995440 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1c93f-6687-4b4e-acd1-70af325e90d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:44:44.995711 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1c93f-6687-4b4e-acd1-70af325e90d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:44:45.005303 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1c93f-6687-4b4e-acd1-70af325e90d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:44:45.006709 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1c93f-6687-4b4e-acd1-70af325e90d0 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:44:45.015847 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1274/2567] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:44:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:44:49.331454 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:44:55.011703 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=58 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:45:09.834589 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1265d71-d651-4449-8058-b7e4c93059cb service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:09.897173 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1265d71-d651-4449-8058-b7e4c93059cb service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:09.904990 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff740a57-b0a5-421a-ae6b-918615edbbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:09.905722 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff740a57-b0a5-421a-ae6b-918615edbbd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:09.905992 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff740a57-b0a5-421a-ae6b-918615edbbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:09.906261 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff740a57-b0a5-421a-ae6b-918615edbbd4 admin admin] PATH_INFO: `/v3/users/b22b4c04eeea48baa623160930fdb3d6` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:09.907385 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff740a57-b0a5-421a-ae6b-918615edbbd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b22b4c04eeea48baa623160930fdb3d6)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:09.909364 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff740a57-b0a5-421a-ae6b-918615edbbd4 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:09.955780 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ff740a57-b0a5-421a-ae6b-918615edbbd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b22b4c04eeea48baa623160930fdb3d6'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:09.956416 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ff740a57-b0a5-421a-ae6b-918615edbbd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b22b4c04eeea48baa623160930fdb3d6'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:09.964909 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ff740a57-b0a5-421a-ae6b-918615edbbd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b22b4c04eeea48baa623160930fdb3d6'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:09.976484 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ff740a57-b0a5-421a-ae6b-918615edbbd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b22b4c04eeea48baa623160930fdb3d6'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:09.977116 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ff740a57-b0a5-421a-ae6b-918615edbbd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b22b4c04eeea48baa623160930fdb3d6'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:09.989616 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1294/2568] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:45:09 2022] DELETE /identity/v3/users/b22b4c04eeea48baa623160930fdb3d6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:45:09.994611 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6ee19-bf07-4754-abe4-0c0b4fea1510 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:09.995109 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6ee19-bf07-4754-abe4-0c0b4fea1510 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:09.995212 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6ee19-bf07-4754-abe4-0c0b4fea1510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:10.014253 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-16a6ee19-bf07-4754-abe4-0c0b4fea1510 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:45:10.015454 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-16a6ee19-bf07-4754-abe4-0c0b4fea1510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:10.074436 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1275/2569] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:45:10.083971 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff740a57-b0a5-421a-ae6b-918615edbbd4 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:10.085422 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ff740a57-b0a5-421a-ae6b-918615edbbd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:10.142762 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff740a57-b0a5-421a-ae6b-918615edbbd4 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:10.146398 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7656c-466b-4cc2-88f8-802288d9bca0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:10.147007 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7656c-466b-4cc2-88f8-802288d9bca0 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:10.147286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7656c-466b-4cc2-88f8-802288d9bca0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:10.147559 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7656c-466b-4cc2-88f8-802288d9bca0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:10.157014 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7656c-466b-4cc2-88f8-802288d9bca0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:10.158434 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7656c-466b-4cc2-88f8-802288d9bca0 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:10.168319 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1295/2570] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:45:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:45:10.429947 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a6ee19-bf07-4754-abe4-0c0b4fea1510 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:10.431532 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-16a6ee19-bf07-4754-abe4-0c0b4fea1510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:10.444336 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a6ee19-bf07-4754-abe4-0c0b4fea1510 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:10.449027 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765aa17-ab6f-49a9-bce6-3169d9085756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:10.449813 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765aa17-ab6f-49a9-bce6-3169d9085756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:10.449943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765aa17-ab6f-49a9-bce6-3169d9085756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:10.450189 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765aa17-ab6f-49a9-bce6-3169d9085756 admin admin] PATH_INFO: `/v3/projects/ae121132e9634b93accbef6500c11681` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:10.451473 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d765aa17-ab6f-49a9-bce6-3169d9085756 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae121132e9634b93accbef6500c11681)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:10.453403 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d765aa17-ab6f-49a9-bce6-3169d9085756 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:10.499940 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d765aa17-ab6f-49a9-bce6-3169d9085756 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae121132e9634b93accbef6500c11681', 'request_id': 'req-d765aa17-ab6f-49a9-bce6-3169d9085756'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:10.505604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d765aa17-ab6f-49a9-bce6-3169d9085756 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae121132e9634b93accbef6500c11681', 'request_id': 'req-d765aa17-ab6f-49a9-bce6-3169d9085756'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:10.512309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d765aa17-ab6f-49a9-bce6-3169d9085756 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae121132e9634b93accbef6500c11681', 'request_id': 'req-d765aa17-ab6f-49a9-bce6-3169d9085756'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:10.524092 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d765aa17-ab6f-49a9-bce6-3169d9085756 admin admin] The token cache is being invalidate because project ae121132e9634b93accbef6500c11681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:45:10.524972 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d765aa17-ab6f-49a9-bce6-3169d9085756 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:10.525575 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d765aa17-ab6f-49a9-bce6-3169d9085756 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:10.527027 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1276/2571] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:45:10 2022] DELETE /identity/v3/projects/ae121132e9634b93accbef6500c11681 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:45:10.530966 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7656c-466b-4cc2-88f8-802288d9bca0 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:10.540262 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7656c-466b-4cc2-88f8-802288d9bca0 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:10.558010 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:45:10.597324 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b657c57-6d9d-49d1-9386-080e7925acf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:10.598027 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b657c57-6d9d-49d1-9386-080e7925acf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:10.598302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b657c57-6d9d-49d1-9386-080e7925acf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:10.598566 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b657c57-6d9d-49d1-9386-080e7925acf9 admin admin] PATH_INFO: `/v3/users/3405adc2883c4267864d189e0157d758` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:10.599661 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b657c57-6d9d-49d1-9386-080e7925acf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3405adc2883c4267864d189e0157d758)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:10.601591 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b657c57-6d9d-49d1-9386-080e7925acf9 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:10.645705 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-0b657c57-6d9d-49d1-9386-080e7925acf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3405adc2883c4267864d189e0157d758'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:10.646340 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-0b657c57-6d9d-49d1-9386-080e7925acf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3405adc2883c4267864d189e0157d758'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:10.654120 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-0b657c57-6d9d-49d1-9386-080e7925acf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3405adc2883c4267864d189e0157d758'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:10.666255 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-0b657c57-6d9d-49d1-9386-080e7925acf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3405adc2883c4267864d189e0157d758'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:10.666895 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-0b657c57-6d9d-49d1-9386-080e7925acf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3405adc2883c4267864d189e0157d758'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:10.678844 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1296/2572] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:45:10 2022] DELETE /identity/v3/users/3405adc2883c4267864d189e0157d758 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:45:10.945046 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765aa17-ab6f-49a9-bce6-3169d9085756 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:10.947133 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d765aa17-ab6f-49a9-bce6-3169d9085756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:10.960456 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765aa17-ab6f-49a9-bce6-3169d9085756 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:11.019255 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d12b3-d9bc-45ef-91cc-cee0cec8f47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:11.019936 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d12b3-d9bc-45ef-91cc-cee0cec8f47b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:11.020130 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d12b3-d9bc-45ef-91cc-cee0cec8f47b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:11.020397 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d12b3-d9bc-45ef-91cc-cee0cec8f47b admin admin] PATH_INFO: `/v3/projects/80afbba338c34792a86e670f7c777ccd` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:11.021529 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d12b3-d9bc-45ef-91cc-cee0cec8f47b admin admin] RBAC: Authorizing `identity:delete_project(project_id=80afbba338c34792a86e670f7c777ccd)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:11.023412 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d12b3-d9bc-45ef-91cc-cee0cec8f47b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:11.055107 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:45:11.072440 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-086d12b3-d9bc-45ef-91cc-cee0cec8f47b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80afbba338c34792a86e670f7c777ccd', 'request_id': 'req-086d12b3-d9bc-45ef-91cc-cee0cec8f47b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:11.077922 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-086d12b3-d9bc-45ef-91cc-cee0cec8f47b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80afbba338c34792a86e670f7c777ccd', 'request_id': 'req-086d12b3-d9bc-45ef-91cc-cee0cec8f47b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:11.084403 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-086d12b3-d9bc-45ef-91cc-cee0cec8f47b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80afbba338c34792a86e670f7c777ccd', 'request_id': 'req-086d12b3-d9bc-45ef-91cc-cee0cec8f47b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:11.098704 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-086d12b3-d9bc-45ef-91cc-cee0cec8f47b admin admin] The token cache is being invalidate because project 80afbba338c34792a86e670f7c777ccd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:45:11.098994 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-086d12b3-d9bc-45ef-91cc-cee0cec8f47b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:11.099610 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-086d12b3-d9bc-45ef-91cc-cee0cec8f47b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:11.101121 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1277/2573] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:45:10 2022] DELETE /identity/v3/projects/80afbba338c34792a86e670f7c777ccd => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:45:11.106592 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe66bc-dee4-4812-b329-cd95e5df6ccb None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:11.107066 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe66bc-dee4-4812-b329-cd95e5df6ccb None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:11.107166 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe66bc-dee4-4812-b329-cd95e5df6ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:11.127077 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-aafe66bc-dee4-4812-b329-cd95e5df6ccb None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:45:11.128267 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-aafe66bc-dee4-4812-b329-cd95e5df6ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:11.186473 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1297/2574] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:45:11.194192 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11238938-fd2a-4478-a247-624801047129 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:11.194427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11238938-fd2a-4478-a247-624801047129 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:11.194702 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11238938-fd2a-4478-a247-624801047129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:11.214579 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-11238938-fd2a-4478-a247-624801047129 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:45:11.215771 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-11238938-fd2a-4478-a247-624801047129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:11.224025 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1278/2575] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:45:11.228926 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe66bc-dee4-4812-b329-cd95e5df6ccb None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:11.238348 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe66bc-dee4-4812-b329-cd95e5df6ccb None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:11.242828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f878cc-bd2d-439d-ac26-71f30e6482d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:11.243585 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f878cc-bd2d-439d-ac26-71f30e6482d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:11.243885 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f878cc-bd2d-439d-ac26-71f30e6482d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:11.244152 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f878cc-bd2d-439d-ac26-71f30e6482d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:11.244818 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f878cc-bd2d-439d-ac26-71f30e6482d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:45:11.245119 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f878cc-bd2d-439d-ac26-71f30e6482d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:11.246696 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f878cc-bd2d-439d-ac26-71f30e6482d5 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:11.258124 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1298/2576] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:45:11.262708 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11238938-fd2a-4478-a247-624801047129 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:11.271828 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11238938-fd2a-4478-a247-624801047129 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:11.277275 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6714d99-3c80-47ff-a089-583d1bae74c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:11.277873 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6714d99-3c80-47ff-a089-583d1bae74c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:11.278153 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6714d99-3c80-47ff-a089-583d1bae74c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:11.278425 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6714d99-3c80-47ff-a089-583d1bae74c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:11.279049 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6714d99-3c80-47ff-a089-583d1bae74c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:11.280996 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6714d99-3c80-47ff-a089-583d1bae74c0 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:11.308202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1279/2577] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:45:11 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:45:11.312458 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f878cc-bd2d-439d-ac26-71f30e6482d5 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:11.322838 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f878cc-bd2d-439d-ac26-71f30e6482d5 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:11.348976 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:45:11.382106 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178248d-30da-4778-8cdf-9c88474ae8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:11.382763 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178248d-30da-4778-8cdf-9c88474ae8f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:11.383042 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178248d-30da-4778-8cdf-9c88474ae8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:11.383315 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178248d-30da-4778-8cdf-9c88474ae8f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:11.384561 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178248d-30da-4778-8cdf-9c88474ae8f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:11.386585 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178248d-30da-4778-8cdf-9c88474ae8f5 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:11.407195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1299/2578] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:45:11 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:45:11.411290 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6714d99-3c80-47ff-a089-583d1bae74c0 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:11.421857 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6714d99-3c80-47ff-a089-583d1bae74c0 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:11.426633 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508f9d0-7e99-4e43-a07f-d9eac8db7e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:11.427233 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508f9d0-7e99-4e43-a07f-d9eac8db7e1b admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:11.427508 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508f9d0-7e99-4e43-a07f-d9eac8db7e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:11.427774 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508f9d0-7e99-4e43-a07f-d9eac8db7e1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:11.428427 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508f9d0-7e99-4e43-a07f-d9eac8db7e1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:11.430123 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508f9d0-7e99-4e43-a07f-d9eac8db7e1b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:11.440744 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1280/2579] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:45:11.444576 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178248d-30da-4778-8cdf-9c88474ae8f5 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:11.455801 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178248d-30da-4778-8cdf-9c88474ae8f5 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:11.460402 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4b876-58b1-4408-a8c1-bf6a195371ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:11.461049 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4b876-58b1-4408-a8c1-bf6a195371ee admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:11.461344 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4b876-58b1-4408-a8c1-bf6a195371ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:11.461610 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4b876-58b1-4408-a8c1-bf6a195371ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:11.462828 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4b876-58b1-4408-a8c1-bf6a195371ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:11.464418 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4b876-58b1-4408-a8c1-bf6a195371ee admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:11.474428 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1300/2580] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:45:11.478321 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508f9d0-7e99-4e43-a07f-d9eac8db7e1b admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:11.488376 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508f9d0-7e99-4e43-a07f-d9eac8db7e1b admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:11.493123 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315000f7-bd00-4877-a0a4-d3c0e8e4b067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:11.493840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315000f7-bd00-4877-a0a4-d3c0e8e4b067 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:11.494118 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315000f7-bd00-4877-a0a4-d3c0e8e4b067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:11.494388 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315000f7-bd00-4877-a0a4-d3c0e8e4b067 admin admin] PATH_INFO: `/v3/projects/4e0cb5f1da364d349ee17b0dd697616d/users/870cb635d36641e4a3d00624d95ba37b/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:11.510875 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315000f7-bd00-4877-a0a4-d3c0e8e4b067 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e0cb5f1da364d349ee17b0dd697616d, user_id=870cb635d36641e4a3d00624d95ba37b, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:11.513572 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315000f7-bd00-4877-a0a4-d3c0e8e4b067 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:11.527104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1281/2581] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:45:11 2022] PUT /identity/v3/projects/4e0cb5f1da364d349ee17b0dd697616d/users/870cb635d36641e4a3d00624d95ba37b/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:45:11.537221 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aec897-37cb-461c-825b-b2b0bbac2e94 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:11.537508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aec897-37cb-461c-825b-b2b0bbac2e94 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:11.537781 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aec897-37cb-461c-825b-b2b0bbac2e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:11.565695 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a0aec897-37cb-461c-825b-b2b0bbac2e94 None None] MFA Rules not processed for user `870cb635d36641e4a3d00624d95ba37b`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:45:11.566895 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a0aec897-37cb-461c-825b-b2b0bbac2e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:11.630603 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1301/2582] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:45:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:45:11.635853 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315000f7-bd00-4877-a0a4-d3c0e8e4b067 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:11.695943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315000f7-bd00-4877-a0a4-d3c0e8e4b067 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:11.701068 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d710b3-ec29-4858-9d8f-cf3d4973f3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:11.701663 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d710b3-ec29-4858-9d8f-cf3d4973f3b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:11.701942 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d710b3-ec29-4858-9d8f-cf3d4973f3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:11.702216 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d710b3-ec29-4858-9d8f-cf3d4973f3b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:11.702872 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d710b3-ec29-4858-9d8f-cf3d4973f3b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:11.704766 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d710b3-ec29-4858-9d8f-cf3d4973f3b1 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:11.731604 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1282/2583] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:45:11 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:45:11.734490 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aec897-37cb-461c-825b-b2b0bbac2e94 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:11.745496 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aec897-37cb-461c-825b-b2b0bbac2e94 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:11.800182 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e95a8-ca66-4fef-ad96-3c121d5deb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:11.800831 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e95a8-ca66-4fef-ad96-3c121d5deb4c admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:11.801107 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e95a8-ca66-4fef-ad96-3c121d5deb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:11.801375 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e95a8-ca66-4fef-ad96-3c121d5deb4c admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:11.801997 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e95a8-ca66-4fef-ad96-3c121d5deb4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:11.803906 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e95a8-ca66-4fef-ad96-3c121d5deb4c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:11.825259 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1302/2584] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:45:11 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:45:11.829214 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d710b3-ec29-4858-9d8f-cf3d4973f3b1 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:11.839338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d710b3-ec29-4858-9d8f-cf3d4973f3b1 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:11.843936 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02543e-407d-40ce-9dd2-e1d6a93fd3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:11.844527 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02543e-407d-40ce-9dd2-e1d6a93fd3c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:11.844843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02543e-407d-40ce-9dd2-e1d6a93fd3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:11.845117 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02543e-407d-40ce-9dd2-e1d6a93fd3c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:11.845750 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f02543e-407d-40ce-9dd2-e1d6a93fd3c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:11.847325 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f02543e-407d-40ce-9dd2-e1d6a93fd3c8 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:11.857800 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1283/2585] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:45:11.861437 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e95a8-ca66-4fef-ad96-3c121d5deb4c admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:11.871727 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e95a8-ca66-4fef-ad96-3c121d5deb4c admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:11.876458 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd8ed58-5954-44b5-9ce3-d5165e39e505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:11.877075 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8ed58-5954-44b5-9ce3-d5165e39e505 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:11.877351 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8ed58-5954-44b5-9ce3-d5165e39e505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:11.877616 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8ed58-5954-44b5-9ce3-d5165e39e505 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:11.878238 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8ed58-5954-44b5-9ce3-d5165e39e505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:11.879800 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8ed58-5954-44b5-9ce3-d5165e39e505 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:11.889995 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1303/2586] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:45:11.893901 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02543e-407d-40ce-9dd2-e1d6a93fd3c8 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:11.905279 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02543e-407d-40ce-9dd2-e1d6a93fd3c8 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:11.910151 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd57426-6c0a-416f-a95c-080df302556f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:11.910803 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd57426-6c0a-416f-a95c-080df302556f admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:11.911081 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd57426-6c0a-416f-a95c-080df302556f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:11.911355 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd57426-6c0a-416f-a95c-080df302556f admin admin] PATH_INFO: `/v3/projects/5ecd8d26e000481eb1be525de3f78fb9/users/c551be71bbad4645ab2d60e1123918cc/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:11.926001 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd57426-6c0a-416f-a95c-080df302556f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ecd8d26e000481eb1be525de3f78fb9, user_id=c551be71bbad4645ab2d60e1123918cc, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:11.928613 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd57426-6c0a-416f-a95c-080df302556f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:11.937618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:45:11.942470 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1284/2587] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:45:11 2022] PUT /identity/v3/projects/5ecd8d26e000481eb1be525de3f78fb9/users/c551be71bbad4645ab2d60e1123918cc/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 06:45:11.954739 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c9c25-17f1-422f-b9e0-febe741163af None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:11.954955 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c9c25-17f1-422f-b9e0-febe741163af None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:11.955251 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c9c25-17f1-422f-b9e0-febe741163af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:11.982714 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-6c0c9c25-17f1-422f-b9e0-febe741163af None None] MFA Rules not processed for user `c551be71bbad4645ab2d60e1123918cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:45:11.983912 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6c0c9c25-17f1-422f-b9e0-febe741163af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:11.987757 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:45:12.053385 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1304/2588] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:45:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:45:12.058478 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd57426-6c0a-416f-a95c-080df302556f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:12.119793 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd57426-6c0a-416f-a95c-080df302556f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:12.124437 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfdfb7-36f6-43d6-bc63-1713e667e385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:12.125403 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfdfb7-36f6-43d6-bc63-1713e667e385 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:12.125796 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfdfb7-36f6-43d6-bc63-1713e667e385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:12.126077 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfdfb7-36f6-43d6-bc63-1713e667e385 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:12.126715 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cfdfb7-36f6-43d6-bc63-1713e667e385 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:12.128615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cfdfb7-36f6-43d6-bc63-1713e667e385 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:12.154440 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1285/2589] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:45:12 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:45:12.158564 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c9c25-17f1-422f-b9e0-febe741163af None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:12.169882 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c9c25-17f1-422f-b9e0-febe741163af None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:12.226138 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d0064-1847-4dcf-99a2-2ce105e75347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:12.226662 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d0064-1847-4dcf-99a2-2ce105e75347 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:12.226940 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d0064-1847-4dcf-99a2-2ce105e75347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:12.227208 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d0064-1847-4dcf-99a2-2ce105e75347 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:12.228364 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5d0064-1847-4dcf-99a2-2ce105e75347 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:12.230302 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5d0064-1847-4dcf-99a2-2ce105e75347 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:12.251997 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1305/2590] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:45:12 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:45:12.256000 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfdfb7-36f6-43d6-bc63-1713e667e385 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:12.266158 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfdfb7-36f6-43d6-bc63-1713e667e385 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:12.270701 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406a4eb-cf1a-4316-ad50-a843732afa79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:12.271287 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406a4eb-cf1a-4316-ad50-a843732afa79 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:12.271884 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406a4eb-cf1a-4316-ad50-a843732afa79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:12.272161 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406a4eb-cf1a-4316-ad50-a843732afa79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:12.272851 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c406a4eb-cf1a-4316-ad50-a843732afa79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:12.274440 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c406a4eb-cf1a-4316-ad50-a843732afa79 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:12.284710 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1286/2591] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:45:12.289623 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d0064-1847-4dcf-99a2-2ce105e75347 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:12.302484 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d0064-1847-4dcf-99a2-2ce105e75347 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:12.307836 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d1e22-7ba7-40ab-a5a7-218626f41d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:12.308518 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d1e22-7ba7-40ab-a5a7-218626f41d58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:12.308832 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d1e22-7ba7-40ab-a5a7-218626f41d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:12.309100 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d1e22-7ba7-40ab-a5a7-218626f41d58 admin admin] PATH_INFO: `/v3/projects/223a41aa496b42c2a82271504d18d2ef/users/59870356aa644567a6b46acef7068165/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:12.325544 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d1e22-7ba7-40ab-a5a7-218626f41d58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=223a41aa496b42c2a82271504d18d2ef, user_id=59870356aa644567a6b46acef7068165, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:12.328648 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d1e22-7ba7-40ab-a5a7-218626f41d58 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:12.344235 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1306/2592] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:45:12 2022] PUT /identity/v3/projects/223a41aa496b42c2a82271504d18d2ef/users/59870356aa644567a6b46acef7068165/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:45:12.348240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406a4eb-cf1a-4316-ad50-a843732afa79 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:12.358670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406a4eb-cf1a-4316-ad50-a843732afa79 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:12.417589 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba00053-dd44-4eed-a81b-55103fcd509d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:12.418202 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba00053-dd44-4eed-a81b-55103fcd509d admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:12.418481 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba00053-dd44-4eed-a81b-55103fcd509d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:12.418751 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba00053-dd44-4eed-a81b-55103fcd509d admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:12.419383 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba00053-dd44-4eed-a81b-55103fcd509d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:12.421005 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba00053-dd44-4eed-a81b-55103fcd509d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:12.430972 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1287/2593] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:45:12.434871 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d1e22-7ba7-40ab-a5a7-218626f41d58 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:12.445083 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d1e22-7ba7-40ab-a5a7-218626f41d58 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:12.450139 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1a92a-f572-40f4-9c20-fe5f2b003a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:12.450865 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1a92a-f572-40f4-9c20-fe5f2b003a5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:12.451171 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1a92a-f572-40f4-9c20-fe5f2b003a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:12.451442 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1a92a-f572-40f4-9c20-fe5f2b003a5e admin admin] PATH_INFO: `/v3/projects/223a41aa496b42c2a82271504d18d2ef/users/59870356aa644567a6b46acef7068165/roles/d8debbb193834fac9cc25f81726f3a67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:12.453468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1a92a-f572-40f4-9c20-fe5f2b003a5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=223a41aa496b42c2a82271504d18d2ef, user_id=59870356aa644567a6b46acef7068165, role_id=d8debbb193834fac9cc25f81726f3a67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:12.456135 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1a92a-f572-40f4-9c20-fe5f2b003a5e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:12.466747 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1307/2594] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:45:12 2022] PUT /identity/v3/projects/223a41aa496b42c2a82271504d18d2ef/users/59870356aa644567a6b46acef7068165/roles/d8debbb193834fac9cc25f81726f3a67 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:45:12.476816 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a523b-8eda-497d-b52e-e45031172f11 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:12.477044 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a523b-8eda-497d-b52e-e45031172f11 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:12.477338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a523b-8eda-497d-b52e-e45031172f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:12.505868 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-243a523b-8eda-497d-b52e-e45031172f11 None None] MFA Rules not processed for user `59870356aa644567a6b46acef7068165`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:45:12.507061 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-243a523b-8eda-497d-b52e-e45031172f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:12.566127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:45:12.576664 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1288/2595] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:45:12 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:45:12.587431 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1a92a-f572-40f4-9c20-fe5f2b003a5e admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:12.589113 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5ba1a92a-f572-40f4-9c20-fe5f2b003a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:12.646622 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1a92a-f572-40f4-9c20-fe5f2b003a5e admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:12.650374 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29981a30-a7e7-4b09-a441-83bc743a1627 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:12.650972 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29981a30-a7e7-4b09-a441-83bc743a1627 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:12.651248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29981a30-a7e7-4b09-a441-83bc743a1627 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:12.651522 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29981a30-a7e7-4b09-a441-83bc743a1627 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:12.660361 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29981a30-a7e7-4b09-a441-83bc743a1627 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:12.661826 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29981a30-a7e7-4b09-a441-83bc743a1627 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:12.666752 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:45:12.673055 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1308/2596] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:45:12 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:45:13.160770 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a523b-8eda-497d-b52e-e45031172f11 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:13.162170 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-243a523b-8eda-497d-b52e-e45031172f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:13.174575 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a523b-8eda-497d-b52e-e45031172f11 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:13.178095 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39a516-141b-47ad-8bcc-c507322c9ebd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:13.178685 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39a516-141b-47ad-8bcc-c507322c9ebd service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:13.178959 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39a516-141b-47ad-8bcc-c507322c9ebd service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:13.179222 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39a516-141b-47ad-8bcc-c507322c9ebd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:13.236959 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff39a516-141b-47ad-8bcc-c507322c9ebd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:13.238420 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff39a516-141b-47ad-8bcc-c507322c9ebd service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:13.248157 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1289/2597] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:45:13 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:45:17.255352 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29981a30-a7e7-4b09-a441-83bc743a1627 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:17.264882 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29981a30-a7e7-4b09-a441-83bc743a1627 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:17.268431 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c1f34-8b8a-4313-95db-5df6991237e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:17.269044 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c1f34-8b8a-4313-95db-5df6991237e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:17.269319 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c1f34-8b8a-4313-95db-5df6991237e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:17.269581 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c1f34-8b8a-4313-95db-5df6991237e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:17.326467 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c1f34-8b8a-4313-95db-5df6991237e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:17.327896 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c1f34-8b8a-4313-95db-5df6991237e5 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:17.336646 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1309/2598] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:45:17 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:45:19.398384 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39a516-141b-47ad-8bcc-c507322c9ebd service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:19.459477 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39a516-141b-47ad-8bcc-c507322c9ebd service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:19.464494 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91c68c-9199-41e4-87fa-0a7e71465bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:19.465149 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91c68c-9199-41e4-87fa-0a7e71465bf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:19.465425 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91c68c-9199-41e4-87fa-0a7e71465bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:19.465693 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91c68c-9199-41e4-87fa-0a7e71465bf0 admin admin] PATH_INFO: `/v3/users/870cb635d36641e4a3d00624d95ba37b` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:19.466788 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91c68c-9199-41e4-87fa-0a7e71465bf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=870cb635d36641e4a3d00624d95ba37b)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:19.468688 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91c68c-9199-41e4-87fa-0a7e71465bf0 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:19.515287 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7e91c68c-9199-41e4-87fa-0a7e71465bf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '870cb635d36641e4a3d00624d95ba37b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:19.515915 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7e91c68c-9199-41e4-87fa-0a7e71465bf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '870cb635d36641e4a3d00624d95ba37b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:19.522200 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7e91c68c-9199-41e4-87fa-0a7e71465bf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '870cb635d36641e4a3d00624d95ba37b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:19.533926 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7e91c68c-9199-41e4-87fa-0a7e71465bf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '870cb635d36641e4a3d00624d95ba37b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:19.541059 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-7e91c68c-9199-41e4-87fa-0a7e71465bf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '870cb635d36641e4a3d00624d95ba37b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:19.546747 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1290/2599] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:45:19 2022] DELETE /identity/v3/users/870cb635d36641e4a3d00624d95ba37b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:45:19.551149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10350903-5352-426d-a3b0-7162189819c6 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:19.551623 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10350903-5352-426d-a3b0-7162189819c6 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:19.551717 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10350903-5352-426d-a3b0-7162189819c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:19.571239 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-10350903-5352-426d-a3b0-7162189819c6 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:45:19.572421 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-10350903-5352-426d-a3b0-7162189819c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:19.631057 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1310/2600] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:45:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:45:19.641296 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91c68c-9199-41e4-87fa-0a7e71465bf0 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:19.642709 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7e91c68c-9199-41e4-87fa-0a7e71465bf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:19.657488 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:45:19.702391 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91c68c-9199-41e4-87fa-0a7e71465bf0 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:19.706117 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e6171-30f5-4464-a5ad-c9ab3fd09e8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:19.706720 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e6171-30f5-4464-a5ad-c9ab3fd09e8b service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:19.706994 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e6171-30f5-4464-a5ad-c9ab3fd09e8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:19.707262 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e6171-30f5-4464-a5ad-c9ab3fd09e8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:19.716746 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e6171-30f5-4464-a5ad-c9ab3fd09e8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:19.718165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e6171-30f5-4464-a5ad-c9ab3fd09e8b service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:19.727408 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1291/2601] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:45:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:45:19.990226 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10350903-5352-426d-a3b0-7162189819c6 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:19.991719 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-10350903-5352-426d-a3b0-7162189819c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:20.005846 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10350903-5352-426d-a3b0-7162189819c6 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:20.010770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16090a-819e-4a21-8b57-bd688d1b1aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:20.011692 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16090a-819e-4a21-8b57-bd688d1b1aac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:20.011927 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16090a-819e-4a21-8b57-bd688d1b1aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:20.012158 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16090a-819e-4a21-8b57-bd688d1b1aac admin admin] PATH_INFO: `/v3/projects/4e0cb5f1da364d349ee17b0dd697616d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:20.013377 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16090a-819e-4a21-8b57-bd688d1b1aac admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e0cb5f1da364d349ee17b0dd697616d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:20.015278 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16090a-819e-4a21-8b57-bd688d1b1aac admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:20.063623 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-0f16090a-819e-4a21-8b57-bd688d1b1aac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e0cb5f1da364d349ee17b0dd697616d', 'request_id': 'req-0f16090a-819e-4a21-8b57-bd688d1b1aac'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:20.069151 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-0f16090a-819e-4a21-8b57-bd688d1b1aac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e0cb5f1da364d349ee17b0dd697616d', 'request_id': 'req-0f16090a-819e-4a21-8b57-bd688d1b1aac'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:20.079432 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-0f16090a-819e-4a21-8b57-bd688d1b1aac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e0cb5f1da364d349ee17b0dd697616d', 'request_id': 'req-0f16090a-819e-4a21-8b57-bd688d1b1aac'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:20.093326 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-0f16090a-819e-4a21-8b57-bd688d1b1aac admin admin] The token cache is being invalidate because project 4e0cb5f1da364d349ee17b0dd697616d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:45:20.093616 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-0f16090a-819e-4a21-8b57-bd688d1b1aac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:20.094258 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-0f16090a-819e-4a21-8b57-bd688d1b1aac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:20.095740 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1311/2602] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:45:19 2022] DELETE /identity/v3/projects/4e0cb5f1da364d349ee17b0dd697616d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:45:20.099739 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e6171-30f5-4464-a5ad-c9ab3fd09e8b service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:20.110277 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e6171-30f5-4464-a5ad-c9ab3fd09e8b service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:20.165061 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128a801-4826-4aa9-9d96-2dc565251a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:20.166328 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128a801-4826-4aa9-9d96-2dc565251a21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:20.166607 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128a801-4826-4aa9-9d96-2dc565251a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:20.166877 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128a801-4826-4aa9-9d96-2dc565251a21 admin admin] PATH_INFO: `/v3/users/c551be71bbad4645ab2d60e1123918cc` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:20.168016 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5128a801-4826-4aa9-9d96-2dc565251a21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c551be71bbad4645ab2d60e1123918cc)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:20.169955 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5128a801-4826-4aa9-9d96-2dc565251a21 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:20.215792 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5128a801-4826-4aa9-9d96-2dc565251a21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c551be71bbad4645ab2d60e1123918cc'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:20.216442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5128a801-4826-4aa9-9d96-2dc565251a21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c551be71bbad4645ab2d60e1123918cc'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:20.224184 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5128a801-4826-4aa9-9d96-2dc565251a21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c551be71bbad4645ab2d60e1123918cc'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:20.238448 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5128a801-4826-4aa9-9d96-2dc565251a21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c551be71bbad4645ab2d60e1123918cc'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:20.245409 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5128a801-4826-4aa9-9d96-2dc565251a21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c551be71bbad4645ab2d60e1123918cc'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:20.251442 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1292/2603] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:45:20 2022] DELETE /identity/v3/users/c551be71bbad4645ab2d60e1123918cc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:45:20.525959 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16090a-819e-4a21-8b57-bd688d1b1aac admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:20.527425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-0f16090a-819e-4a21-8b57-bd688d1b1aac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:20.542126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16090a-819e-4a21-8b57-bd688d1b1aac admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:20.561358 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:45:20.602733 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9aa04e-b03a-4c02-85b3-76a993275faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:20.603239 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9aa04e-b03a-4c02-85b3-76a993275faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:20.603508 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9aa04e-b03a-4c02-85b3-76a993275faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:20.603772 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9aa04e-b03a-4c02-85b3-76a993275faa admin admin] PATH_INFO: `/v3/projects/5ecd8d26e000481eb1be525de3f78fb9` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:20.605038 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9aa04e-b03a-4c02-85b3-76a993275faa admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ecd8d26e000481eb1be525de3f78fb9)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:20.606921 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9aa04e-b03a-4c02-85b3-76a993275faa admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:20.656227 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ae9aa04e-b03a-4c02-85b3-76a993275faa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ecd8d26e000481eb1be525de3f78fb9', 'request_id': 'req-ae9aa04e-b03a-4c02-85b3-76a993275faa'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:20.662019 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ae9aa04e-b03a-4c02-85b3-76a993275faa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ecd8d26e000481eb1be525de3f78fb9', 'request_id': 'req-ae9aa04e-b03a-4c02-85b3-76a993275faa'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:20.668197 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ae9aa04e-b03a-4c02-85b3-76a993275faa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ecd8d26e000481eb1be525de3f78fb9', 'request_id': 'req-ae9aa04e-b03a-4c02-85b3-76a993275faa'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:20.680396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ae9aa04e-b03a-4c02-85b3-76a993275faa admin admin] The token cache is being invalidate because project 5ecd8d26e000481eb1be525de3f78fb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:45:20.681218 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ae9aa04e-b03a-4c02-85b3-76a993275faa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:20.681884 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ae9aa04e-b03a-4c02-85b3-76a993275faa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:20.683344 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1312/2604] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:45:20 2022] DELETE /identity/v3/projects/5ecd8d26e000481eb1be525de3f78fb9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:45:20.687306 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128a801-4826-4aa9-9d96-2dc565251a21 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:20.688721 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5128a801-4826-4aa9-9d96-2dc565251a21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:20.702173 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128a801-4826-4aa9-9d96-2dc565251a21 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:20.735779 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:45:20.762406 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469e98a-ba96-4eca-9b27-1210c44cfe05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:20.763034 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469e98a-ba96-4eca-9b27-1210c44cfe05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:20.763311 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469e98a-ba96-4eca-9b27-1210c44cfe05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:20.763610 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469e98a-ba96-4eca-9b27-1210c44cfe05 admin admin] PATH_INFO: `/v3/users/59870356aa644567a6b46acef7068165` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:20.764720 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469e98a-ba96-4eca-9b27-1210c44cfe05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59870356aa644567a6b46acef7068165)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:20.766615 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469e98a-ba96-4eca-9b27-1210c44cfe05 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:20.811712 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0469e98a-ba96-4eca-9b27-1210c44cfe05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59870356aa644567a6b46acef7068165'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:20.812347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0469e98a-ba96-4eca-9b27-1210c44cfe05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59870356aa644567a6b46acef7068165'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:20.819469 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0469e98a-ba96-4eca-9b27-1210c44cfe05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59870356aa644567a6b46acef7068165'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:20.831910 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0469e98a-ba96-4eca-9b27-1210c44cfe05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59870356aa644567a6b46acef7068165'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:20.838724 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0469e98a-ba96-4eca-9b27-1210c44cfe05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59870356aa644567a6b46acef7068165'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:20.844402 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1293/2605] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:45:20 2022] DELETE /identity/v3/users/59870356aa644567a6b46acef7068165 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:45:21.115165 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9aa04e-b03a-4c02-85b3-76a993275faa admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:21.116644 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ae9aa04e-b03a-4c02-85b3-76a993275faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:21.130857 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9aa04e-b03a-4c02-85b3-76a993275faa admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:21.186964 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a16bf-a3bf-48ce-b02a-e7c637e3d1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:21.187586 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a16bf-a3bf-48ce-b02a-e7c637e3d1fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:21.187858 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a16bf-a3bf-48ce-b02a-e7c637e3d1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:21.188133 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a16bf-a3bf-48ce-b02a-e7c637e3d1fe admin admin] PATH_INFO: `/v3/projects/223a41aa496b42c2a82271504d18d2ef` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:21.189248 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a16bf-a3bf-48ce-b02a-e7c637e3d1fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=223a41aa496b42c2a82271504d18d2ef)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:21.191167 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a16bf-a3bf-48ce-b02a-e7c637e3d1fe admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:21.239026 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d91a16bf-a3bf-48ce-b02a-e7c637e3d1fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '223a41aa496b42c2a82271504d18d2ef', 'request_id': 'req-d91a16bf-a3bf-48ce-b02a-e7c637e3d1fe'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:21.242717 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:45:21.245783 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d91a16bf-a3bf-48ce-b02a-e7c637e3d1fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '223a41aa496b42c2a82271504d18d2ef', 'request_id': 'req-d91a16bf-a3bf-48ce-b02a-e7c637e3d1fe'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:21.253277 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d91a16bf-a3bf-48ce-b02a-e7c637e3d1fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '223a41aa496b42c2a82271504d18d2ef', 'request_id': 'req-d91a16bf-a3bf-48ce-b02a-e7c637e3d1fe'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:21.265727 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d91a16bf-a3bf-48ce-b02a-e7c637e3d1fe admin admin] The token cache is being invalidate because project 223a41aa496b42c2a82271504d18d2ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:45:21.266013 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d91a16bf-a3bf-48ce-b02a-e7c637e3d1fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:21.266627 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d91a16bf-a3bf-48ce-b02a-e7c637e3d1fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:21.268106 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1313/2606] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:45:21 2022] DELETE /identity/v3/projects/223a41aa496b42c2a82271504d18d2ef => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:45:21.274028 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb5258-03a2-416d-8847-9a5558b517ec None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:21.274315 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb5258-03a2-416d-8847-9a5558b517ec None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:21.274587 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb5258-03a2-416d-8847-9a5558b517ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:21.294450 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-82bb5258-03a2-416d-8847-9a5558b517ec None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:45:21.295743 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-82bb5258-03a2-416d-8847-9a5558b517ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:21.356840 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1294/2607] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:45:21.366314 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6816c77-8b57-4aab-bd63-1a3cb5cd8035 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:21.366536 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6816c77-8b57-4aab-bd63-1a3cb5cd8035 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:21.366807 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6816c77-8b57-4aab-bd63-1a3cb5cd8035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:21.386790 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-f6816c77-8b57-4aab-bd63-1a3cb5cd8035 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:45:21.387994 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f6816c77-8b57-4aab-bd63-1a3cb5cd8035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:21.396052 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1314/2608] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:45:21.400097 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb5258-03a2-416d-8847-9a5558b517ec None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:21.410876 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb5258-03a2-416d-8847-9a5558b517ec None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:21.415474 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9c421-8a76-403a-ba5f-0a133e87700c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:21.416785 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9c421-8a76-403a-ba5f-0a133e87700c admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:21.417067 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9c421-8a76-403a-ba5f-0a133e87700c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:21.417345 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9c421-8a76-403a-ba5f-0a133e87700c admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:21.417984 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9c421-8a76-403a-ba5f-0a133e87700c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:45:21.418285 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9c421-8a76-403a-ba5f-0a133e87700c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:21.419873 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9c421-8a76-403a-ba5f-0a133e87700c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:21.431372 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1295/2609] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:45:21.435587 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6816c77-8b57-4aab-bd63-1a3cb5cd8035 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:21.445476 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6816c77-8b57-4aab-bd63-1a3cb5cd8035 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:21.450395 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd81802-5c3d-4542-8f42-0d680aec5708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:21.450991 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd81802-5c3d-4542-8f42-0d680aec5708 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:21.451290 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd81802-5c3d-4542-8f42-0d680aec5708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:21.451535 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd81802-5c3d-4542-8f42-0d680aec5708 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:21.452159 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd81802-5c3d-4542-8f42-0d680aec5708 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:21.454069 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd81802-5c3d-4542-8f42-0d680aec5708 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:21.481109 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1315/2610] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:45:21 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:45:21.485315 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9c421-8a76-403a-ba5f-0a133e87700c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:21.495204 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9c421-8a76-403a-ba5f-0a133e87700c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:21.535080 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:45:21.553645 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756099e-dbed-4876-8b48-4846fef780f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:21.554246 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756099e-dbed-4876-8b48-4846fef780f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:21.554521 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756099e-dbed-4876-8b48-4846fef780f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:21.554788 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756099e-dbed-4876-8b48-4846fef780f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:21.555406 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756099e-dbed-4876-8b48-4846fef780f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:21.557500 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756099e-dbed-4876-8b48-4846fef780f3 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:21.577746 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1296/2611] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:45:21 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:45:21.581681 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd81802-5c3d-4542-8f42-0d680aec5708 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:21.592318 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd81802-5c3d-4542-8f42-0d680aec5708 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:21.596963 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9207c17-0269-49f4-98f7-6bb4bbcf357f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:21.597551 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9207c17-0269-49f4-98f7-6bb4bbcf357f admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:21.597824 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9207c17-0269-49f4-98f7-6bb4bbcf357f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:21.598089 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9207c17-0269-49f4-98f7-6bb4bbcf357f admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:21.598712 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9207c17-0269-49f4-98f7-6bb4bbcf357f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:21.600270 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9207c17-0269-49f4-98f7-6bb4bbcf357f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:21.610165 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1316/2612] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:45:21.613976 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756099e-dbed-4876-8b48-4846fef780f3 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:21.624338 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756099e-dbed-4876-8b48-4846fef780f3 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:21.629755 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91be82e0-fc0d-4f1a-9b48-df695803c09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:21.630347 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be82e0-fc0d-4f1a-9b48-df695803c09a admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:21.630626 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be82e0-fc0d-4f1a-9b48-df695803c09a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:21.630900 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be82e0-fc0d-4f1a-9b48-df695803c09a admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:21.631530 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91be82e0-fc0d-4f1a-9b48-df695803c09a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:21.633141 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91be82e0-fc0d-4f1a-9b48-df695803c09a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:21.643753 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1297/2613] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:45:21.647691 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9207c17-0269-49f4-98f7-6bb4bbcf357f admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:21.657964 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9207c17-0269-49f4-98f7-6bb4bbcf357f admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:21.662568 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651573eb-17a6-40b9-9ae1-a175e52aeee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:21.663200 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651573eb-17a6-40b9-9ae1-a175e52aeee0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:21.663473 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651573eb-17a6-40b9-9ae1-a175e52aeee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:21.663741 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651573eb-17a6-40b9-9ae1-a175e52aeee0 admin admin] PATH_INFO: `/v3/projects/94dc72a3a01144e7a1ab75633601fe03/users/00e143de63dc4f6595b715587a94df7f/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:21.678022 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651573eb-17a6-40b9-9ae1-a175e52aeee0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94dc72a3a01144e7a1ab75633601fe03, user_id=00e143de63dc4f6595b715587a94df7f, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:21.680618 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651573eb-17a6-40b9-9ae1-a175e52aeee0 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:21.693413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1317/2614] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:45:21 2022] PUT /identity/v3/projects/94dc72a3a01144e7a1ab75633601fe03/users/00e143de63dc4f6595b715587a94df7f/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:45:21.704600 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09604ae-6e02-4037-a073-4d2f3a7b8ad3 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:21.705098 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09604ae-6e02-4037-a073-4d2f3a7b8ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:21.705211 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09604ae-6e02-4037-a073-4d2f3a7b8ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:21.733524 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c09604ae-6e02-4037-a073-4d2f3a7b8ad3 None None] MFA Rules not processed for user `00e143de63dc4f6595b715587a94df7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:45:21.734733 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c09604ae-6e02-4037-a073-4d2f3a7b8ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:21.798127 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1298/2615] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:45:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:45:21.810444 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651573eb-17a6-40b9-9ae1-a175e52aeee0 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:21.812117 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-651573eb-17a6-40b9-9ae1-a175e52aeee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:21.867754 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651573eb-17a6-40b9-9ae1-a175e52aeee0 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:21.871970 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09e0b7-fa3b-4b98-9d38-9b7cb8d058ff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dcfe5244bd04c00b6dde1d6f795d4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:21.872566 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09e0b7-fa3b-4b98-9d38-9b7cb8d058ff service glance] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:21.872880 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09e0b7-fa3b-4b98-9d38-9b7cb8d058ff service glance] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:21.873154 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09e0b7-fa3b-4b98-9d38-9b7cb8d058ff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:21.881537 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09e0b7-fa3b-4b98-9d38-9b7cb8d058ff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:21.882953 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09e0b7-fa3b-4b98-9d38-9b7cb8d058ff service glance] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:21.891628 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1318/2616] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:45:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:45:31.795198 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=47,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:45:31.887758 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:DELETE=2,keystone:SELECT=58,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:45:39.505962 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09604ae-6e02-4037-a073-4d2f3a7b8ad3 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:39.569446 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09604ae-6e02-4037-a073-4d2f3a7b8ad3 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:39.573868 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87236102-db16-4350-9819-93866b758b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:39.574496 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87236102-db16-4350-9819-93866b758b7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:39.574780 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87236102-db16-4350-9819-93866b758b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:39.575055 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87236102-db16-4350-9819-93866b758b7e admin admin] PATH_INFO: `/v3/users/00e143de63dc4f6595b715587a94df7f` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:39.576655 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87236102-db16-4350-9819-93866b758b7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=00e143de63dc4f6595b715587a94df7f)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:39.578593 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87236102-db16-4350-9819-93866b758b7e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:39.624262 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-87236102-db16-4350-9819-93866b758b7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00e143de63dc4f6595b715587a94df7f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:39.625181 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-87236102-db16-4350-9819-93866b758b7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00e143de63dc4f6595b715587a94df7f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:39.632104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-87236102-db16-4350-9819-93866b758b7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00e143de63dc4f6595b715587a94df7f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:39.644194 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-87236102-db16-4350-9819-93866b758b7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00e143de63dc4f6595b715587a94df7f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:39.650845 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-87236102-db16-4350-9819-93866b758b7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00e143de63dc4f6595b715587a94df7f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:39.657180 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1299/2617] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:45:39 2022] DELETE /identity/v3/users/00e143de63dc4f6595b715587a94df7f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:45:39.661638 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faadc77c-bc27-4386-998e-2a1b8c893292 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:39.662159 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faadc77c-bc27-4386-998e-2a1b8c893292 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:39.662294 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faadc77c-bc27-4386-998e-2a1b8c893292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:39.681232 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-faadc77c-bc27-4386-998e-2a1b8c893292 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:45:39.682435 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-faadc77c-bc27-4386-998e-2a1b8c893292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:39.740468 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1319/2618] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:45:39.750257 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87236102-db16-4350-9819-93866b758b7e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:39.751677 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-87236102-db16-4350-9819-93866b758b7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:39.807236 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87236102-db16-4350-9819-93866b758b7e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:39.811045 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890b769-b4fd-4787-97fb-60dc368b4bc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:39.811640 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890b769-b4fd-4787-97fb-60dc368b4bc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:39.811919 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890b769-b4fd-4787-97fb-60dc368b4bc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:39.812189 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890b769-b4fd-4787-97fb-60dc368b4bc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:39.820936 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890b769-b4fd-4787-97fb-60dc368b4bc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:39.822289 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890b769-b4fd-4787-97fb-60dc368b4bc1 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:39.835309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1300/2619] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:45:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:45:40.094649 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faadc77c-bc27-4386-998e-2a1b8c893292 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:40.096115 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-faadc77c-bc27-4386-998e-2a1b8c893292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:40.108268 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faadc77c-bc27-4386-998e-2a1b8c893292 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:40.114639 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a08c3-6b7a-43db-8484-c61bd7415777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:40.115285 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a08c3-6b7a-43db-8484-c61bd7415777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:40.115582 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a08c3-6b7a-43db-8484-c61bd7415777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:40.115846 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a08c3-6b7a-43db-8484-c61bd7415777 admin admin] PATH_INFO: `/v3/projects/94dc72a3a01144e7a1ab75633601fe03` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:40.116989 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a08c3-6b7a-43db-8484-c61bd7415777 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94dc72a3a01144e7a1ab75633601fe03)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:40.118881 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a08c3-6b7a-43db-8484-c61bd7415777 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:40.166218 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-aa3a08c3-6b7a-43db-8484-c61bd7415777 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94dc72a3a01144e7a1ab75633601fe03', 'request_id': 'req-aa3a08c3-6b7a-43db-8484-c61bd7415777'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:40.171794 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-aa3a08c3-6b7a-43db-8484-c61bd7415777 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94dc72a3a01144e7a1ab75633601fe03', 'request_id': 'req-aa3a08c3-6b7a-43db-8484-c61bd7415777'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:40.177843 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-aa3a08c3-6b7a-43db-8484-c61bd7415777 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94dc72a3a01144e7a1ab75633601fe03', 'request_id': 'req-aa3a08c3-6b7a-43db-8484-c61bd7415777'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:40.190286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-aa3a08c3-6b7a-43db-8484-c61bd7415777 admin admin] The token cache is being invalidate because project 94dc72a3a01144e7a1ab75633601fe03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:45:40.190575 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-aa3a08c3-6b7a-43db-8484-c61bd7415777 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:40.191196 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-aa3a08c3-6b7a-43db-8484-c61bd7415777 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:45:40.192775 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1320/2620] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:45:40 2022] DELETE /identity/v3/projects/94dc72a3a01144e7a1ab75633601fe03 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:45:40.198537 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba28b1-fc32-4519-928b-79f5055fb9b0 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:40.198826 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba28b1-fc32-4519-928b-79f5055fb9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:40.199104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba28b1-fc32-4519-928b-79f5055fb9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:40.219004 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-cdba28b1-fc32-4519-928b-79f5055fb9b0 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:45:40.220203 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-cdba28b1-fc32-4519-928b-79f5055fb9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:40.237618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:45:40.281914 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1301/2621] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:45:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:45:40.289617 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b506b34-0173-4768-ba9a-89b2ea37e317 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:40.289903 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b506b34-0173-4768-ba9a-89b2ea37e317 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:40.290180 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b506b34-0173-4768-ba9a-89b2ea37e317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:40.312114 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-7b506b34-0173-4768-ba9a-89b2ea37e317 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:45:40.313505 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7b506b34-0173-4768-ba9a-89b2ea37e317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:40.321399 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1321/2622] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:45:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:45:40.325647 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba28b1-fc32-4519-928b-79f5055fb9b0 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:40.336008 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba28b1-fc32-4519-928b-79f5055fb9b0 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:40.340748 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be5dcbe-5acc-4a01-993c-cbb88beff054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:40.341295 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5dcbe-5acc-4a01-993c-cbb88beff054 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:40.341576 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5dcbe-5acc-4a01-993c-cbb88beff054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:40.341979 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5dcbe-5acc-4a01-993c-cbb88beff054 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:40.342629 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be5dcbe-5acc-4a01-993c-cbb88beff054 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:45:40.342931 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be5dcbe-5acc-4a01-993c-cbb88beff054 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:40.344514 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be5dcbe-5acc-4a01-993c-cbb88beff054 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:40.355921 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1302/2623] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:45:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:45:40.360785 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b506b34-0173-4768-ba9a-89b2ea37e317 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:40.369907 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b506b34-0173-4768-ba9a-89b2ea37e317 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:40.375326 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7051b05c-ac2d-4478-993a-9d15c18b3985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:40.375918 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051b05c-ac2d-4478-993a-9d15c18b3985 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:40.376196 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051b05c-ac2d-4478-993a-9d15c18b3985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:40.376466 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051b05c-ac2d-4478-993a-9d15c18b3985 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:40.377126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7051b05c-ac2d-4478-993a-9d15c18b3985 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:40.379001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7051b05c-ac2d-4478-993a-9d15c18b3985 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:40.405711 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1322/2624] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:45:40 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:45:40.409818 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be5dcbe-5acc-4a01-993c-cbb88beff054 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:40.420602 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be5dcbe-5acc-4a01-993c-cbb88beff054 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:40.476049 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69b6a7-f9ee-40db-8c5a-0495c507208d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:40.476645 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69b6a7-f9ee-40db-8c5a-0495c507208d admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:40.476952 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69b6a7-f9ee-40db-8c5a-0495c507208d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:40.477222 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69b6a7-f9ee-40db-8c5a-0495c507208d admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:40.477834 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c69b6a7-f9ee-40db-8c5a-0495c507208d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:40.479735 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c69b6a7-f9ee-40db-8c5a-0495c507208d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:40.500890 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1303/2625] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:45:40 2022] POST /identity/v3/users => generated 283 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:45:40.505075 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7051b05c-ac2d-4478-993a-9d15c18b3985 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:40.515933 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7051b05c-ac2d-4478-993a-9d15c18b3985 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:40.520549 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec9dac-2d4b-4a20-ba14-b047f7cf909e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:40.521195 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec9dac-2d4b-4a20-ba14-b047f7cf909e admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:40.521473 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec9dac-2d4b-4a20-ba14-b047f7cf909e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:40.521747 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec9dac-2d4b-4a20-ba14-b047f7cf909e admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:40.522381 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec9dac-2d4b-4a20-ba14-b047f7cf909e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:40.523967 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec9dac-2d4b-4a20-ba14-b047f7cf909e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:40.534522 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1323/2626] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:45:40.538468 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69b6a7-f9ee-40db-8c5a-0495c507208d admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:40.548732 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69b6a7-f9ee-40db-8c5a-0495c507208d admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:40.553394 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2e4d1-6445-4180-9f2d-bf34be896a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:40.553987 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2e4d1-6445-4180-9f2d-bf34be896a24 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:40.554268 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2e4d1-6445-4180-9f2d-bf34be896a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:40.554541 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2e4d1-6445-4180-9f2d-bf34be896a24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:40.555169 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2e4d1-6445-4180-9f2d-bf34be896a24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:40.556828 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2e4d1-6445-4180-9f2d-bf34be896a24 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:40.567282 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1304/2627] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:45:40.571467 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec9dac-2d4b-4a20-ba14-b047f7cf909e admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:40.582200 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec9dac-2d4b-4a20-ba14-b047f7cf909e admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:40.586800 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b78eb-d054-43fa-8ba7-7409eddd7c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:40.587441 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b78eb-d054-43fa-8ba7-7409eddd7c59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:40.587718 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b78eb-d054-43fa-8ba7-7409eddd7c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:40.587995 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b78eb-d054-43fa-8ba7-7409eddd7c59 admin admin] PATH_INFO: `/v3/projects/97fa96bfc5a94561900a783650ba10ac/users/66f1dd68c98d4e29a3dcaa89f10652c8/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:40.602420 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b78eb-d054-43fa-8ba7-7409eddd7c59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97fa96bfc5a94561900a783650ba10ac, user_id=66f1dd68c98d4e29a3dcaa89f10652c8, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:40.605005 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b78eb-d054-43fa-8ba7-7409eddd7c59 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:40.618736 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1324/2628] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:45:40 2022] PUT /identity/v3/projects/97fa96bfc5a94561900a783650ba10ac/users/66f1dd68c98d4e29a3dcaa89f10652c8/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:45:40.630597 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8788c1c-28e4-4ae0-944f-fc731586297b None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:40.630886 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8788c1c-28e4-4ae0-944f-fc731586297b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:40.631163 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8788c1c-28e4-4ae0-944f-fc731586297b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:40.659040 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-e8788c1c-28e4-4ae0-944f-fc731586297b None None] MFA Rules not processed for user `66f1dd68c98d4e29a3dcaa89f10652c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:45:40.660249 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e8788c1c-28e4-4ae0-944f-fc731586297b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:40.724237 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1305/2629] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:45:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:45:40.736103 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b78eb-d054-43fa-8ba7-7409eddd7c59 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:45:40.737817 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d65b78eb-d054-43fa-8ba7-7409eddd7c59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:45:40.759279 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:45:40.796107 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b78eb-d054-43fa-8ba7-7409eddd7c59 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:45:40.799764 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf5951-ffa0-442e-a451-51f903f43f08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:45:40.800359 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf5951-ffa0-442e-a451-51f903f43f08 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:45:40.800638 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf5951-ffa0-442e-a451-51f903f43f08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:45:40.800935 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf5951-ffa0-442e-a451-51f903f43f08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:45:40.809370 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf5951-ffa0-442e-a451-51f903f43f08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:45:40.810879 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf5951-ffa0-442e-a451-51f903f43f08 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:45:40.819897 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1325/2630] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:45:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:45:50.721244 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:45:50.815621 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=17 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:46:16.029119 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8788c1c-28e4-4ae0-944f-fc731586297b None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:46:16.109488 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8788c1c-28e4-4ae0-944f-fc731586297b None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:46:16.113996 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28281120-5df7-4c1f-9f77-e6c68eb61c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:46:16.114618 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28281120-5df7-4c1f-9f77-e6c68eb61c19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:16.114891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28281120-5df7-4c1f-9f77-e6c68eb61c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:16.115160 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28281120-5df7-4c1f-9f77-e6c68eb61c19 admin admin] PATH_INFO: `/v3/users/66f1dd68c98d4e29a3dcaa89f10652c8` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:16.116267 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28281120-5df7-4c1f-9f77-e6c68eb61c19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66f1dd68c98d4e29a3dcaa89f10652c8)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:46:16.118281 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28281120-5df7-4c1f-9f77-e6c68eb61c19 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:46:16.164247 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-28281120-5df7-4c1f-9f77-e6c68eb61c19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66f1dd68c98d4e29a3dcaa89f10652c8'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:46:16.165061 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-28281120-5df7-4c1f-9f77-e6c68eb61c19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66f1dd68c98d4e29a3dcaa89f10652c8'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:46:16.172199 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-28281120-5df7-4c1f-9f77-e6c68eb61c19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66f1dd68c98d4e29a3dcaa89f10652c8'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:46:16.184224 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-28281120-5df7-4c1f-9f77-e6c68eb61c19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66f1dd68c98d4e29a3dcaa89f10652c8'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:46:16.191167 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-28281120-5df7-4c1f-9f77-e6c68eb61c19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66f1dd68c98d4e29a3dcaa89f10652c8'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:46:16.197444 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1306/2631] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:46:16 2022] DELETE /identity/v3/users/66f1dd68c98d4e29a3dcaa89f10652c8 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:46:16.202082 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416205e7-8ea6-4aac-9d0a-572038e35c27 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:16.202677 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416205e7-8ea6-4aac-9d0a-572038e35c27 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:16.202917 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416205e7-8ea6-4aac-9d0a-572038e35c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:16.234237 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-416205e7-8ea6-4aac-9d0a-572038e35c27 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:46:16.235415 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-416205e7-8ea6-4aac-9d0a-572038e35c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:46:16.293064 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1326/2632] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:46:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:46:16.302974 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28281120-5df7-4c1f-9f77-e6c68eb61c19 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:46:16.304452 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-28281120-5df7-4c1f-9f77-e6c68eb61c19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:46:16.369894 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28281120-5df7-4c1f-9f77-e6c68eb61c19 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:46:16.373572 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ee0ea-703e-452a-9147-63ec72821f65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:46:16.374167 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ee0ea-703e-452a-9147-63ec72821f65 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:16.374700 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ee0ea-703e-452a-9147-63ec72821f65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:16.374975 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ee0ea-703e-452a-9147-63ec72821f65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:16.381806 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:46:16.384754 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ee0ea-703e-452a-9147-63ec72821f65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:46:16.386810 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ee0ea-703e-452a-9147-63ec72821f65 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:46:16.397518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1307/2633] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:46:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 06:46:16.656253 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416205e7-8ea6-4aac-9d0a-572038e35c27 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:46:16.658104 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-416205e7-8ea6-4aac-9d0a-572038e35c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:46:16.671751 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416205e7-8ea6-4aac-9d0a-572038e35c27 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:46:16.676835 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda1140-18d8-4e55-a857-dcf449ee7df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:46:16.677462 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda1140-18d8-4e55-a857-dcf449ee7df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:16.677773 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda1140-18d8-4e55-a857-dcf449ee7df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:16.677985 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda1140-18d8-4e55-a857-dcf449ee7df5 admin admin] PATH_INFO: `/v3/projects/97fa96bfc5a94561900a783650ba10ac` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:16.679081 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda1140-18d8-4e55-a857-dcf449ee7df5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97fa96bfc5a94561900a783650ba10ac)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:46:16.681509 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda1140-18d8-4e55-a857-dcf449ee7df5 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:46:16.727856 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cdda1140-18d8-4e55-a857-dcf449ee7df5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97fa96bfc5a94561900a783650ba10ac', 'request_id': 'req-cdda1140-18d8-4e55-a857-dcf449ee7df5'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:46:16.733337 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cdda1140-18d8-4e55-a857-dcf449ee7df5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97fa96bfc5a94561900a783650ba10ac', 'request_id': 'req-cdda1140-18d8-4e55-a857-dcf449ee7df5'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:46:16.739597 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cdda1140-18d8-4e55-a857-dcf449ee7df5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97fa96bfc5a94561900a783650ba10ac', 'request_id': 'req-cdda1140-18d8-4e55-a857-dcf449ee7df5'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:46:16.752476 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cdda1140-18d8-4e55-a857-dcf449ee7df5 admin admin] The token cache is being invalidate because project 97fa96bfc5a94561900a783650ba10ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:46:16.752790 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cdda1140-18d8-4e55-a857-dcf449ee7df5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:46:16.753404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cdda1140-18d8-4e55-a857-dcf449ee7df5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:46:16.754858 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1327/2634] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:46:16 2022] DELETE /identity/v3/projects/97fa96bfc5a94561900a783650ba10ac => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:46:16.760354 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102cc570-c144-40d9-9c32-a5c7dc3bb8b1 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:16.760634 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102cc570-c144-40d9-9c32-a5c7dc3bb8b1 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:16.760943 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102cc570-c144-40d9-9c32-a5c7dc3bb8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:16.779766 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-102cc570-c144-40d9-9c32-a5c7dc3bb8b1 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:46:16.780989 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-102cc570-c144-40d9-9c32-a5c7dc3bb8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:46:16.839804 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1308/2635] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:46:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 06:46:16.847471 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e6e58-384f-4dc6-a261-813528c0e647 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:16.847750 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e6e58-384f-4dc6-a261-813528c0e647 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:16.848016 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e6e58-384f-4dc6-a261-813528c0e647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:16.868481 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-1e1e6e58-384f-4dc6-a261-813528c0e647 None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:46:16.869698 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1e1e6e58-384f-4dc6-a261-813528c0e647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:46:16.877752 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1328/2636] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:46:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:46:16.881914 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102cc570-c144-40d9-9c32-a5c7dc3bb8b1 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:46:16.891794 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102cc570-c144-40d9-9c32-a5c7dc3bb8b1 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:46:16.896301 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d8262-e791-4a5f-b0d4-437ab2b4f7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:46:16.896958 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d8262-e791-4a5f-b0d4-437ab2b4f7c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:16.897240 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d8262-e791-4a5f-b0d4-437ab2b4f7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:16.897514 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d8262-e791-4a5f-b0d4-437ab2b4f7c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:16.898156 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d8262-e791-4a5f-b0d4-437ab2b4f7c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 06:46:16.898489 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d8262-e791-4a5f-b0d4-437ab2b4f7c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:46:16.900104 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d8262-e791-4a5f-b0d4-437ab2b4f7c2 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:46:16.911582 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1309/2637] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Oct 20 06:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 06:46:16.916126 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e6e58-384f-4dc6-a261-813528c0e647 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:46:16.925210 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e6e58-384f-4dc6-a261-813528c0e647 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:46:16.929721 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f3644-9454-4118-ae0b-75320a99f87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:46:16.930346 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f3644-9454-4118-ae0b-75320a99f87b admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:16.930620 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f3644-9454-4118-ae0b-75320a99f87b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:16.930887 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f3644-9454-4118-ae0b-75320a99f87b admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:16.931519 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f3644-9454-4118-ae0b-75320a99f87b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:46:16.933432 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f3644-9454-4118-ae0b-75320a99f87b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:46:16.959899 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1329/2638] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Oct 20 06:46:16 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 06:46:16.964591 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d8262-e791-4a5f-b0d4-437ab2b4f7c2 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:46:16.974530 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d8262-e791-4a5f-b0d4-437ab2b4f7c2 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:46:17.030684 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991062bc-5619-4fc2-a048-03f04419c350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:46:17.031332 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991062bc-5619-4fc2-a048-03f04419c350 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:17.031606 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991062bc-5619-4fc2-a048-03f04419c350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:17.031881 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991062bc-5619-4fc2-a048-03f04419c350 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:17.032517 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991062bc-5619-4fc2-a048-03f04419c350 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:46:17.034504 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991062bc-5619-4fc2-a048-03f04419c350 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:46:17.059802 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1310/2639] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Oct 20 06:46:16 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 06:46:17.061066 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f3644-9454-4118-ae0b-75320a99f87b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:46:17.071816 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f3644-9454-4118-ae0b-75320a99f87b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:46:17.076451 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f4de1-3f59-4d21-8084-f7323088e90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:46:17.077085 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f4de1-3f59-4d21-8084-f7323088e90d admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:17.077367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f4de1-3f59-4d21-8084-f7323088e90d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:17.077640 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f4de1-3f59-4d21-8084-f7323088e90d admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:17.078277 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f4de1-3f59-4d21-8084-f7323088e90d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:46:17.079852 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f4de1-3f59-4d21-8084-f7323088e90d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:46:17.090442 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1330/2640] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 06:46:17.094291 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991062bc-5619-4fc2-a048-03f04419c350 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:46:17.104207 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991062bc-5619-4fc2-a048-03f04419c350 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:46:17.108749 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d37f70-0364-4157-9432-3582a1c6be96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:46:17.109330 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d37f70-0364-4157-9432-3582a1c6be96 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:17.109608 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d37f70-0364-4157-9432-3582a1c6be96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:17.109876 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d37f70-0364-4157-9432-3582a1c6be96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:17.110499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d37f70-0364-4157-9432-3582a1c6be96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:46:17.112059 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d37f70-0364-4157-9432-3582a1c6be96 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:46:17.122518 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1311/2641] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Oct 20 06:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 06:46:17.127413 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f4de1-3f59-4d21-8084-f7323088e90d admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:46:17.137246 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f4de1-3f59-4d21-8084-f7323088e90d admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:46:17.141836 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd456f9c-9a50-49d4-a660-78cd9e52e0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:46:17.142475 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd456f9c-9a50-49d4-a660-78cd9e52e0c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:17.142757 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd456f9c-9a50-49d4-a660-78cd9e52e0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:17.143031 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd456f9c-9a50-49d4-a660-78cd9e52e0c4 admin admin] PATH_INFO: `/v3/projects/ce72a4f6216b4f428718648e0ec619c1/users/49b139ebf6e546d5a359cef570ff8e21/roles/c1e0df53e1264ec79d91065e25acd29d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:17.158030 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd456f9c-9a50-49d4-a660-78cd9e52e0c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce72a4f6216b4f428718648e0ec619c1, user_id=49b139ebf6e546d5a359cef570ff8e21, role_id=c1e0df53e1264ec79d91065e25acd29d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:46:17.160647 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd456f9c-9a50-49d4-a660-78cd9e52e0c4 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:46:17.172994 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1331/2642] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Oct 20 06:46:17 2022] PUT /identity/v3/projects/ce72a4f6216b4f428718648e0ec619c1/users/49b139ebf6e546d5a359cef570ff8e21/roles/c1e0df53e1264ec79d91065e25acd29d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:46:17.183049 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f43fb-1944-460b-9348-364f26d52cfd None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:17.183351 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f43fb-1944-460b-9348-364f26d52cfd None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:17.183627 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f43fb-1944-460b-9348-364f26d52cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:17.211330 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-004f43fb-1944-460b-9348-364f26d52cfd None None] MFA Rules not processed for user `49b139ebf6e546d5a359cef570ff8e21`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:46:17.213158 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-004f43fb-1944-460b-9348-364f26d52cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:46:17.260165 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:46:17.279020 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1312/2643] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:46:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:46:17.290240 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd456f9c-9a50-49d4-a660-78cd9e52e0c4 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:46:17.291977 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-dd456f9c-9a50-49d4-a660-78cd9e52e0c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:46:17.303806 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:46:17.360320 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd456f9c-9a50-49d4-a660-78cd9e52e0c4 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:46:17.364163 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc3489-cd63-47df-b1ac-7e4b3498eac2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb00a74a99e47c1ac938f07d26a4bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:46:17.364808 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc3489-cd63-47df-b1ac-7e4b3498eac2 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:17.365083 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc3489-cd63-47df-b1ac-7e4b3498eac2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:17.365350 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc3489-cd63-47df-b1ac-7e4b3498eac2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:17.373367 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccc3489-cd63-47df-b1ac-7e4b3498eac2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:46:17.375033 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccc3489-cd63-47df-b1ac-7e4b3498eac2 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:46:17.383444 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1332/2644] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:46:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:46:18.737714 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39e98d-219d-49ad-ae0b-0df4d535687a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:18.738235 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39e98d-219d-49ad-ae0b-0df4d535687a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:18.738235 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39e98d-219d-49ad-ae0b-0df4d535687a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:18.757391 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-9d39e98d-219d-49ad-ae0b-0df4d535687a None None] MFA Rules not processed for user `49b139ebf6e546d5a359cef570ff8e21`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:46:18.758709 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-9d39e98d-219d-49ad-ae0b-0df4d535687a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:46:18.765567 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1313/2645] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:46:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:46:19.261724 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc3489-cd63-47df-b1ac-7e4b3498eac2 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:46:19.263271 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7ccc3489-cd63-47df-b1ac-7e4b3498eac2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:46:19.331994 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc3489-cd63-47df-b1ac-7e4b3498eac2 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:46:19.335747 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5566897-5ed7-44f9-9dfc-4d96490e7c26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:46:19.336430 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5566897-5ed7-44f9-9dfc-4d96490e7c26 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:19.336811 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5566897-5ed7-44f9-9dfc-4d96490e7c26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:19.337091 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5566897-5ed7-44f9-9dfc-4d96490e7c26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:19.338937 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a5566897-5ed7-44f9-9dfc-4d96490e7c26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:46:19.422376 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5566897-5ed7-44f9-9dfc-4d96490e7c26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:46:19.423803 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5566897-5ed7-44f9-9dfc-4d96490e7c26 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:46:19.446210 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1333/2646] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:46:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:46:21.780062 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39e98d-219d-49ad-ae0b-0df4d535687a None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:46:21.782241 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-9d39e98d-219d-49ad-ae0b-0df4d535687a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:46:21.803168 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39e98d-219d-49ad-ae0b-0df4d535687a None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:46:21.811913 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4bd274-85af-4893-b8c0-8e42c8d7b744 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae2c1c886cd94d9f90082f66e7e8c9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:46:21.812891 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4bd274-85af-4893-b8c0-8e42c8d7b744 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:21.813400 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4bd274-85af-4893-b8c0-8e42c8d7b744 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:21.814372 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4bd274-85af-4893-b8c0-8e42c8d7b744 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:21.822499 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1b4bd274-85af-4893-b8c0-8e42c8d7b744 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:46:21.836012 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4bd274-85af-4893-b8c0-8e42c8d7b744 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:46:21.838747 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4bd274-85af-4893-b8c0-8e42c8d7b744 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:46:21.851172 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1314/2647] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:46:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:46:29.443076 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=87 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:46:31.846063 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=13 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:46:34.972805 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e08802-80e1-4700-b9d3-f1ddfc92fd4a None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:34.973286 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e08802-80e1-4700-b9d3-f1ddfc92fd4a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:34.973418 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e08802-80e1-4700-b9d3-f1ddfc92fd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:34.992731 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-54e08802-80e1-4700-b9d3-f1ddfc92fd4a None None] MFA Rules not processed for user `7a59f547a20e48ee88e3e620ba1e87f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:46:34.993917 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-54e08802-80e1-4700-b9d3-f1ddfc92fd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:46:35.049429 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1334/2648] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Oct 20 06:46:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:46:35.094594 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4bd274-85af-4893-b8c0-8e42c8d7b744 service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:46:35.095999 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1b4bd274-85af-4893-b8c0-8e42c8d7b744 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:46:35.163463 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4bd274-85af-4893-b8c0-8e42c8d7b744 service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:46:35.169138 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b51f6e-e48e-4384-bb33-a92723f9a576 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeeab506ab5c4d6386fc899086f0abf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:46:35.169739 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b51f6e-e48e-4384-bb33-a92723f9a576 service swift] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:35.170022 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b51f6e-e48e-4384-bb33-a92723f9a576 service swift] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:35.170296 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b51f6e-e48e-4384-bb33-a92723f9a576 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:35.178208 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b51f6e-e48e-4384-bb33-a92723f9a576 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:46:35.180367 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b51f6e-e48e-4384-bb33-a92723f9a576 service swift] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:46:35.188265 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1315/2649] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Oct 20 06:46:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 06:46:35.698710 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9b830-b060-4aa2-84a3-47bba6136c09 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:35.699001 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9b830-b060-4aa2-84a3-47bba6136c09 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:35.699280 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9b830-b060-4aa2-84a3-47bba6136c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:35.719404 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-60f9b830-b060-4aa2-84a3-47bba6136c09 None None] MFA Rules not processed for user `7a59f547a20e48ee88e3e620ba1e87f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:46:35.720754 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-60f9b830-b060-4aa2-84a3-47bba6136c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:46:35.728542 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1335/2650] 10.222.0.20 () {58 vars in 967 bytes} [Thu Oct 20 06:46:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:46:35.741168 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b51f6e-e48e-4384-bb33-a92723f9a576 service swift] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:46:35.752670 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b51f6e-e48e-4384-bb33-a92723f9a576 service swift] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:46:35.757873 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e5c20-8cd1-4d4e-97e0-156bae12536b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeeab506ab5c4d6386fc899086f0abf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:46:35.758303 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e5c20-8cd1-4d4e-97e0-156bae12536b service swift] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:35.758584 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e5c20-8cd1-4d4e-97e0-156bae12536b service swift] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:35.758856 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e5c20-8cd1-4d4e-97e0-156bae12536b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:35.766632 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e5c20-8cd1-4d4e-97e0-156bae12536b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:46:35.768201 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e5c20-8cd1-4d4e-97e0-156bae12536b service swift] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:46:35.776048 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1316/2651] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Oct 20 06:46:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 06:46:36.000950 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710b41a-990f-43df-a763-256bbaab4b2d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:36.001419 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710b41a-990f-43df-a763-256bbaab4b2d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:36.001522 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710b41a-990f-43df-a763-256bbaab4b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:36.021357 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d710b41a-990f-43df-a763-256bbaab4b2d None None] MFA Rules not processed for user `7a59f547a20e48ee88e3e620ba1e87f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:46:36.022555 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d710b41a-990f-43df-a763-256bbaab4b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:46:36.030102 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1336/2652] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Oct 20 06:46:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:46:36.042683 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e5c20-8cd1-4d4e-97e0-156bae12536b service swift] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:46:36.053180 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e5c20-8cd1-4d4e-97e0-156bae12536b service swift] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:46:36.058201 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df2b3c-b852-4df0-971e-f04b5dfbab57 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeeab506ab5c4d6386fc899086f0abf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:46:36.058788 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df2b3c-b852-4df0-971e-f04b5dfbab57 service swift] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:36.059069 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df2b3c-b852-4df0-971e-f04b5dfbab57 service swift] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:36.059345 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df2b3c-b852-4df0-971e-f04b5dfbab57 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:36.067225 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df2b3c-b852-4df0-971e-f04b5dfbab57 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:46:36.068824 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df2b3c-b852-4df0-971e-f04b5dfbab57 service swift] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:46:36.076971 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1317/2653] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Oct 20 06:46:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 06:46:36.404149 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174b9bc-3f34-4d5b-ab29-4b34e1bcdf68 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:36.404449 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174b9bc-3f34-4d5b-ab29-4b34e1bcdf68 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:36.404747 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174b9bc-3f34-4d5b-ab29-4b34e1bcdf68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:36.424425 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-f174b9bc-3f34-4d5b-ab29-4b34e1bcdf68 None None] MFA Rules not processed for user `7a59f547a20e48ee88e3e620ba1e87f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:46:36.425678 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f174b9bc-3f34-4d5b-ab29-4b34e1bcdf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:46:36.433752 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1337/2654] 10.222.0.20 () {58 vars in 967 bytes} [Thu Oct 20 06:46:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:46:36.445697 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df2b3c-b852-4df0-971e-f04b5dfbab57 service swift] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:46:36.456386 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df2b3c-b852-4df0-971e-f04b5dfbab57 service swift] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:46:36.461315 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb9415-03bb-44bd-8930-062200967532 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeeab506ab5c4d6386fc899086f0abf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:46:36.461904 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb9415-03bb-44bd-8930-062200967532 service swift] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:36.462186 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb9415-03bb-44bd-8930-062200967532 service swift] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:36.462462 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb9415-03bb-44bd-8930-062200967532 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:36.470272 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb9415-03bb-44bd-8930-062200967532 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:46:36.471835 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb9415-03bb-44bd-8930-062200967532 service swift] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:46:36.480273 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1318/2655] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Oct 20 06:46:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 06:46:38.620343 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174b9bc-3f34-4d5b-ab29-4b34e1bcdf68 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:46:38.681793 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174b9bc-3f34-4d5b-ab29-4b34e1bcdf68 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:46:38.686206 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976c78b-d871-419f-b097-4a4b1b8c1154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:46:38.686821 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976c78b-d871-419f-b097-4a4b1b8c1154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:38.687094 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976c78b-d871-419f-b097-4a4b1b8c1154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:38.687360 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976c78b-d871-419f-b097-4a4b1b8c1154 admin admin] PATH_INFO: `/v3/users/49b139ebf6e546d5a359cef570ff8e21` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:38.688459 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976c78b-d871-419f-b097-4a4b1b8c1154 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49b139ebf6e546d5a359cef570ff8e21)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:46:38.690396 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976c78b-d871-419f-b097-4a4b1b8c1154 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:46:38.735512 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a976c78b-d871-419f-b097-4a4b1b8c1154 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49b139ebf6e546d5a359cef570ff8e21'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:46:38.736162 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a976c78b-d871-419f-b097-4a4b1b8c1154 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49b139ebf6e546d5a359cef570ff8e21'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:46:38.744117 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a976c78b-d871-419f-b097-4a4b1b8c1154 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49b139ebf6e546d5a359cef570ff8e21'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:46:38.756272 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a976c78b-d871-419f-b097-4a4b1b8c1154 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49b139ebf6e546d5a359cef570ff8e21'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:46:38.756908 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a976c78b-d871-419f-b097-4a4b1b8c1154 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49b139ebf6e546d5a359cef570ff8e21'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:46:38.768610 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1338/2656] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Oct 20 06:46:38 2022] DELETE /identity/v3/users/49b139ebf6e546d5a359cef570ff8e21 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:46:38.773443 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f9498-0573-49b3-a8d5-05d0631eeb4c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:38.773951 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f9498-0573-49b3-a8d5-05d0631eeb4c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:38.774090 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f9498-0573-49b3-a8d5-05d0631eeb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:38.793682 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-fe7f9498-0573-49b3-a8d5-05d0631eeb4c None None] MFA Rules not processed for user `6f317beb484644609d07d666527f760d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 06:46:38.794872 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fe7f9498-0573-49b3-a8d5-05d0631eeb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:46:38.856711 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1319/2657] 10.222.0.20 () {58 vars in 958 bytes} [Thu Oct 20 06:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 06:46:38.866317 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976c78b-d871-419f-b097-4a4b1b8c1154 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:46:38.868363 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a976c78b-d871-419f-b097-4a4b1b8c1154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:46:38.875276 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:46:38.927182 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976c78b-d871-419f-b097-4a4b1b8c1154 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:46:38.931426 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1393054-041a-43bd-ac09-0c808625b644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072ce66e7174837aeb8c7898026fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:46:38.932039 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1393054-041a-43bd-ac09-0c808625b644 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:38.932294 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1393054-041a-43bd-ac09-0c808625b644 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:38.932560 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1393054-041a-43bd-ac09-0c808625b644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:38.941240 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1393054-041a-43bd-ac09-0c808625b644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:46:38.942652 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1393054-041a-43bd-ac09-0c808625b644 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:46:38.951927 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1339/2658] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:46:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:46:39.117936 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f9498-0573-49b3-a8d5-05d0631eeb4c None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:46:39.119497 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fe7f9498-0573-49b3-a8d5-05d0631eeb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:46:39.133562 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f9498-0573-49b3-a8d5-05d0631eeb4c None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:46:39.138208 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd905a2-8dbc-48a1-8cfb-0a2acd3d56e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f317beb484644609d07d666527f760d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edda8534137f405d818dbc74b7391c73', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:46:39.138843 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd905a2-8dbc-48a1-8cfb-0a2acd3d56e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:46:39.139135 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd905a2-8dbc-48a1-8cfb-0a2acd3d56e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:46:39.139392 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd905a2-8dbc-48a1-8cfb-0a2acd3d56e7 admin admin] PATH_INFO: `/v3/projects/ce72a4f6216b4f428718648e0ec619c1` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:46:39.141358 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd905a2-8dbc-48a1-8cfb-0a2acd3d56e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce72a4f6216b4f428718648e0ec619c1)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:46:39.143205 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd905a2-8dbc-48a1-8cfb-0a2acd3d56e7 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:46:39.187074 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:46:39.190314 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-dcd905a2-8dbc-48a1-8cfb-0a2acd3d56e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce72a4f6216b4f428718648e0ec619c1', 'request_id': 'req-dcd905a2-8dbc-48a1-8cfb-0a2acd3d56e7'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:46:39.196135 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-dcd905a2-8dbc-48a1-8cfb-0a2acd3d56e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce72a4f6216b4f428718648e0ec619c1', 'request_id': 'req-dcd905a2-8dbc-48a1-8cfb-0a2acd3d56e7'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:46:39.202327 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-dcd905a2-8dbc-48a1-8cfb-0a2acd3d56e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce72a4f6216b4f428718648e0ec619c1', 'request_id': 'req-dcd905a2-8dbc-48a1-8cfb-0a2acd3d56e7'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:46:39.215024 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-dcd905a2-8dbc-48a1-8cfb-0a2acd3d56e7 admin admin] The token cache is being invalidate because project ce72a4f6216b4f428718648e0ec619c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 06:46:39.215309 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-dcd905a2-8dbc-48a1-8cfb-0a2acd3d56e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:46:39.215919 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-dcd905a2-8dbc-48a1-8cfb-0a2acd3d56e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 06:46:39.217441 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1320/2659] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Oct 20 06:46:39 2022] DELETE /identity/v3/projects/ce72a4f6216b4f428718648e0ec619c1 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 06:46:48.947772 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=24 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:46:49.212054 dv-cinder-861566-3-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 06:49:14.345087 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1393054-041a-43bd-ac09-0c808625b644 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 06:49:14.346469 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b1393054-041a-43bd-ac09-0c808625b644 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:49:14.414770 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1393054-041a-43bd-ac09-0c808625b644 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 06:49:14.472005 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff0059-0f59-48c0-8d3e-d29d7160f59a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfbaaf32475a4c82a34948602d8f4e35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee65b9a8229495bbe97318f060d6342', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 06:49:14.472599 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff0059-0f59-48c0-8d3e-d29d7160f59a service placement] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 06:49:14.472898 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff0059-0f59-48c0-8d3e-d29d7160f59a service placement] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 06:49:14.473166 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff0059-0f59-48c0-8d3e-d29d7160f59a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 06:49:14.474851 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6fff0059-0f59-48c0-8d3e-d29d7160f59a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 06:49:14.485921 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff0059-0f59-48c0-8d3e-d29d7160f59a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 06:49:14.487486 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff0059-0f59-48c0-8d3e-d29d7160f59a service placement] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 06:49:14.498073 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1340/2660] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Oct 20 06:49:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 06:49:24.493669 dv-cinder-861566-3-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=58 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}